-- Logs begin at Tue 2022-08-23 06:27:44 UTC, end at Tue 2022-08-23 07:34:36 UTC. -- Aug 23 06:58:12.357725 dv-cinder-854092-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 23 06:58:12.366600 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 23 06:58:12.368636 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 23 06:58:12 2022] *** Aug 23 06:58:12.368636 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 23 06:58:12.368636 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 23 06:58:12.368636 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: nodename: dv-cinder-854092-1-iscsi Aug 23 06:58:12.368636 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: machine: x86_64 Aug 23 06:58:12.368636 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: clock source: unix Aug 23 06:58:12.368636 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: pcre jit disabled Aug 23 06:58:12.368636 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: detected number of CPU cores: 4 Aug 23 06:58:12.368636 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: current working directory: / Aug 23 06:58:12.368636 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: detected binary path: /usr/bin/uwsgi-core Aug 23 06:58:12.368636 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: your processes number limit is 47680 Aug 23 06:58:12.369201 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: your memory page size is 4096 bytes Aug 23 06:58:12.369201 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: detected max file descriptor number: 2048 Aug 23 06:58:12.369201 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: lock engine: pthread robust mutexes Aug 23 06:58:12.369201 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: thunder lock: enabled Aug 23 06:58:12.369201 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 23 06:58:12.369201 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 23 06:58:12.408373 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: Python main interpreter initialized at 0x55d4138bd9c0 Aug 23 06:58:12.408373 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: python threads support enabled Aug 23 06:58:12.408373 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: your server socket listen backlog is limited to 100 connections Aug 23 06:58:12.408373 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: your mercy for graceful operations on workers is 90 seconds Aug 23 06:58:12.408735 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: mapped 403077 bytes (393 KB) for 2 cores Aug 23 06:58:12.408817 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: *** Operational MODE: preforking *** Aug 23 06:58:12.409064 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: *** uWSGI is running in multiple interpreter mode *** Aug 23 06:58:12.409064 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: spawned uWSGI master process (pid: 81702) Aug 23 06:58:12.409270 dv-cinder-854092-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 23 06:58:12.410755 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: spawned uWSGI worker 1 (pid: 81714, cores: 1) Aug 23 06:58:12.410755 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: spawned uWSGI worker 2 (pid: 81715, cores: 1) Aug 23 06:58:12.410755 dv-cinder-854092-1-iscsi devstack@keystone.service[81702]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 23 06:58:16.680087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 06:58:16.680087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: from cryptography.utils import int_from_bytes Aug 23 06:58:16.680759 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 06:58:16.680759 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: from cryptography.utils import int_from_bytes Aug 23 06:58:16.681169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 06:58:16.681169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: from cryptography.utils import int_from_bytes Aug 23 06:58:16.681275 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 06:58:16.681275 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: from cryptography.utils import int_from_bytes Aug 23 06:58:16.730327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 23 06:58:16.730781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 23 06:58:16.730875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 23 06:58:16.730965 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 23 06:58:16.731037 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] command line args: [] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 23 06:58:16.731095 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] command line args: [] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 23 06:58:16.731169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 23 06:58:16.731251 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 23 06:58:16.731325 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ================================================================================ {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 23 06:58:16.731397 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ================================================================================ {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 23 06:58:16.731500 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] admin_token = **** {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.731572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] admin_token = **** {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.731649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] config_dir = [] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.731713 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] config_dir = [] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.731848 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.731979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.732052 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] config_source = [] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.732126 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] config_source = [] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.732186 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] debug = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.732250 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] debug = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.732397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.732608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.732705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.732766 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.732889 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.732968 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.733155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.733284 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.733359 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.733420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.733617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] list_limit = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.733742 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] list_limit = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.733742 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] log_config_append = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.733871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] log_config_append = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.733936 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.734000 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.734143 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] log_dir = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.734221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] log_dir = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.734288 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] log_file = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.734357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] log_file = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.734478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.734558 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.734720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.734801 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.734982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.735094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.735215 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.735321 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.735396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.735508 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.735594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.735656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.735837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.735920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.736057 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.736151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.736294 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.736375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.736533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.736621 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.736758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.736845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.736982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.737076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.737251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.737368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.737479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.737575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.737741 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.737832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.738246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.738335 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.738416 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] publish_errors = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.738539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] publish_errors = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.738621 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.738692 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.738851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.738941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.739012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.739145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.739258 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.739390 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.739547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.739628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.739749 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] standard_threads = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.739825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] standard_threads = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.739947 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.740025 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.740188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.740279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.740352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.740429 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.740567 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] use_journal = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.740643 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] use_journal = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.740712 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] use_json = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.740781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] use_json = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.740856 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] use_stderr = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.740977 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] use_stderr = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.741061 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] use_syslog = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.741133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] use_syslog = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.741202 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.741272 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 06:58:16.741459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.741541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.741695 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.741777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.741954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.742037 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.742191 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.742279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.742440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.742522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.742672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.742751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.742923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.743010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.743152 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.743234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.743399 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.743502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.743677 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.743766 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.743928 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.744016 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.744158 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.744238 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.744401 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.744483 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.744636 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.744716 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.744877 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.744960 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.745121 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.745204 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.745341 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.745419 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.745592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.745718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.745844 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.745928 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.746236 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.746317 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.746429 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.746517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.746683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.746767 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.746947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.747030 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.747197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.747280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.747467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.747564 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.747763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.747849 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.747921 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.747997 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.748149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.748236 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.748384 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.748461 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.748633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.748716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.748879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.748969 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.749110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.749188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.749340 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.749423 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.749584 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.749669 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.749830 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.749909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.750065 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.750145 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.750311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.750396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.750542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.750624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.750775 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.750852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.751011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.751094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.751239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.751316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.751493 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.751571 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.751723 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.751877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.751957 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.752029 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.752174 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.752254 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.752422 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.752508 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.752650 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.752730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.752885 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.752964 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.753117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.753201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.753347 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.753424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.753582 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.753659 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.753978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.754110 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.754210 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.754317 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.754413 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.754508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.754602 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.754705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.754802 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.754917 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.755025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.755104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.755184 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.755257 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.755377 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.755511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.755650 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.755790 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.755894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.755971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.756110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.756247 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.756401 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.756527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.756612 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.756696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.756808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.756934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.757037 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.757115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.757274 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.757356 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.757512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.757600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.757777 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.757854 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.758016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.758133 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.758261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.758382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.758506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.758584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.758739 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.758818 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.758974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.759057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.759209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.759291 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.759480 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.759563 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.759719 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.759801 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.759955 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.760034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.760200 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.760280 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.760434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.760513 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.760671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.760749 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.760906 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.760987 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.761141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.761227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.761376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.761456 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.761662 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.761747 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.761897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.761977 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.762142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.762225 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.762389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.762468 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.762637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.762716 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.762882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.762968 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.763126 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.763205 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.763370 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.763491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.763632 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.763717 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.763867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.763954 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.764099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.764186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.764335 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.764413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.764567 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.764646 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.764810 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.764895 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.765045 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.765124 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.765293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.765371 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.765531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.765616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.765789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.765866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.766045 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.766127 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.766286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.766372 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.766518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] auth.external = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.766597 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] auth.external = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.766751 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.766829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.767048 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.767156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.767281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.767369 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.767542 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] auth.password = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.767632 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] auth.password = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.767772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] auth.token = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.767850 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] auth.token = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.768014 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.768101 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.768249 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.768336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.768497 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.768578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.768729 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.768814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.768975 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.769054 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.769222 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.769301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.769454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.769541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.769690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] credential.caching = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.769776 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] credential.caching = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.769935 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.770014 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.770186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.770276 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.770430 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.770507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.770664 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.770743 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.770905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.770991 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.771137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.771216 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.771388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.771498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.771664 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.771755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.771901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.771986 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.772151 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.772230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.772433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.772519 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.772678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.772756 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.772937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.773014 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.773178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.773264 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.773432 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.773511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.773670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] federation.caching = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.773755 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] federation.caching = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.773903 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.773988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.774148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.774226 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.774393 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.774472 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.774623 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.774709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.774867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.774946 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.775101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.775180 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.775351 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.775451 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.775612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.775691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.775866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.775944 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.776103 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.776184 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.776347 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.776426 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.776583 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] identity.caching = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.776661 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] identity.caching = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.776838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.776919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.777075 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.777154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.777357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.777443 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.777639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.777719 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.777846 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.778085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.778170 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.778335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.778429 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.778589 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.778668 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.778844 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.778923 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.779082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.779169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.779319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.779398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.779580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.779659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.779816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.779907 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.780063 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.780143 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.780311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.780389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.780556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.780641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.780807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.780888 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.781051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.781128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.781309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.781394 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.781549 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.781627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.781783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.781860 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.782011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.782096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.782247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.782326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.782480 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.782558 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.782715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.782803 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.782955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.783033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.783191 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.783283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.783459 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.783558 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.783691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.783769 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.783938 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.784017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.784175 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.784260 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.784408 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.784486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.784655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.784733 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.784887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.784972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.785120 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.785198 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.785353 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.785431 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.785581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.785666 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.785819 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.785899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.786054 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.786133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.786292 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.786412 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.786535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.786653 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.786789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.786891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.787042 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.787144 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.787280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.787392 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.787528 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.787647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.787781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.787863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.788037 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.788117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.788284 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.788362 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.788524 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.788604 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.788766 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.788845 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.789004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.789083 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.789235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.789314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.789473 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.789613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.789729 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.789854 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.789960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.790109 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.790208 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.790366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.790445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.790622 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.790709 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.790867 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.790946 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.791106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.791186 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.791339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.791444 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.791596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.791681 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.791835 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.791914 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.792065 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.792156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.792309 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.792392 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.792553 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.792632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.792798 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.792884 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.793098 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.793221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.793352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.793576 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.793655 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.793827 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.793906 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.794071 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.794150 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.794314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.794394 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.794553 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.794631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.794791 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.794868 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.795027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.795104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.795274 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.795354 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.795538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.795626 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.795779 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.795856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.796020 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.796105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.796256 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.796335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.796497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.796574 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.796731 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.796809 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.796968 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.797046 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.797204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.797289 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.797450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.797529 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.797688 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.797772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.797918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.797995 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.798155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.798237 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] resource.caching = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.798394 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] resource.caching = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.798472 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.798643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.798721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.798883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.798963 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.799117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.799193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.799364 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.799462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.799628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.799707 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.799892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.799978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.800118 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.800195 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.800351 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.800429 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.800590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] role.caching = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.800668 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.800822 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] role.caching = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.800899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] role.driver = sql {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.801057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.801132 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] role.driver = sql {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.801288 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.801373 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.801849 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.801941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.802095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.802174 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.802342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.802419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.802585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.802669 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.802829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.802908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.803072 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.803150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.803315 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.803401 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.803573 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.803651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.803810 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.803886 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.804055 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.804144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.804299 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.804378 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.804543 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.804621 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.804790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.804931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.805172 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.805248 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.805404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.805481 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.805652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.805738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.805899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.805978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.806142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.806218 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.806379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.806463 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.806610 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.806688 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.806865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.806944 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.807105 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.807183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.807339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.807435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.807599 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.807676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.807830 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.807909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.808061 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.808139 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.808311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.808390 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.808583 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.808667 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.808954 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.809031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.809241 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.809376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.809488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.809720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.809798 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] token.caching = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.809960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.810053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.810202 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] token.caching = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.810300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.810439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.810516 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.810688 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.810764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.810925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.811011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.811163 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.811240 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.811431 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.811512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.811674 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.811763 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.811920 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.812004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.812159 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.812236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.812406 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.812533 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.812691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.812825 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.812952 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.813076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.813202 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.813335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.813462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.813588 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.813712 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.813837 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.813962 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.814099 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.814237 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.814363 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.connection = **** {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.814498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.814630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.814759 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.connection = **** {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.814895 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.815026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.815150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.815282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.815408 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.815553 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.815680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.815805 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.815930 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.816078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.816203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.816330 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.816454 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.816580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.816705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.816840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.816966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.817091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.817215 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.817341 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.817463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.817595 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.817727 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.817862 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.817991 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.818116 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.818240 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.818371 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.818498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.818622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.818747 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.818872 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.818996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.819127 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.819252 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 23 06:58:16.819375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 06:58:16.819375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81715) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 23 06:58:16.847778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:16.848018 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:16.848117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:16.848294 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:16.848430 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:16.848512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:16.856855 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:16.856979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:16.884874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:16.885155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:16.885277 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:16.885447 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:16.938517 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:16.938936 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:16.986514 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:16.986722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:16.988159 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:16.988396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.144568 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.144778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.144875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.144947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.145055 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.145137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.145201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.145263 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.145403 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.145531 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.145615 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.145695 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.168394 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.168590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.168733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.168899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.169022 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.169127 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.169226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.169415 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.169556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.169631 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.169701 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.169810 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.169900 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.170103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.181187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.181688 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.194183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.194316 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.194473 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.194587 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.194655 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.194716 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.194908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.195038 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.195123 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.195197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.195306 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81715) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.195392 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.195585 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.195813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 06:58:17.212535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.212726 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 06:58:17.213072 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.213309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 06:58:17.213586 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.213854 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.214289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.214416 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.214487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 06:58:17.214645 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.214784 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.215021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 06:58:17.215703 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.215703 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.216290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.216642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.226645 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.227032 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.227156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.227370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.227520 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.227684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.228006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.228449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.237364 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81715) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.238078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.240886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81715) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.241013 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.241177 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.241303 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.241429 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.241559 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.241741 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.241868 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.242028 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.242155 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.242237 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.242405 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.242531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.242652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.242778 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.242960 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.243185 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.243313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.243465 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.243600 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.243811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.243936 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.266418 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.266685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.267898 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81715) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.268181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.268307 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.268451 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.272831 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81715) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.273120 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.273244 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.273405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.273549 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.273716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.279893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81715) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.280162 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.280283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.280517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.284271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81715) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.284561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.284684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.284887 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.285010 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.285096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.285164 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.285392 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.285520 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.285659 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.285754 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.285908 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.285997 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.286207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.286333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.286521 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.303696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.303983 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.304292 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.304561 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.307833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81715) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.308486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.311234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.311558 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.311968 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.312272 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.315758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81715) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.316498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.318857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81715) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.319661 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.322092 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.322389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.322656 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.322889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.323014 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.323141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.323222 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.323462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.323585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.323798 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.324092 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.324400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.339628 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81715) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.339923 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.340243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.340465 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.340568 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.340777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.341009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.341280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.347476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.348207 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.349024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.349303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.349749 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.350033 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.351925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81715) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.352212 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.352512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.352691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.352775 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.352971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.353100 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.353285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.353587 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.353862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.364759 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81715) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.365016 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.365319 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.365499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.365589 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.365656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.365869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.366009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.366121 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.366213 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.366432 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.366558 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.366774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.367060 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.381834 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81715) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.382469 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.384869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81715) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.385537 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.387880 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.388467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.389313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.389910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.390762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81715) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.391043 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.391335 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.391480 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.391656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.391979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.398441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.399937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.402199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.402887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.403180 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.403473 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.403796 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.404121 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81715) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.407074 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.407363 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.407674 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.407948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.411754 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.412055 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.412342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.415867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81715) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.416142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.416423 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.419244 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.419564 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.419853 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.420144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.420433 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.420723 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.421008 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.421294 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.421595 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.421886 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.422146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.422429 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.422706 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.422960 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.423618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81715) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 06:58:17.423901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.424183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.424458 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.424729 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.425014 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.425286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.425554 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.425842 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.426123 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.426397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.426659 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.427242 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.427531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81715) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 06:58:17.457504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d4138bd9c0 pid: 81714 (default app) Aug 23 06:58:17.460873 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f189a0-2822-4c87-bc8d-2ce536bf34b7 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:17.461167 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f189a0-2822-4c87-bc8d-2ce536bf34b7 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:17.461293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d4138bd9c0 pid: 81715 (default app) Aug 23 06:58:17.461445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f189a0-2822-4c87-bc8d-2ce536bf34b7 None None] PATH_INFO: `/v3` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:17.465816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Tue Aug 23 06:58:17 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:24.080328 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23950b88-d55f-40bd-ac27-08148ed8afe3 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:24.081098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23950b88-d55f-40bd-ac27-08148ed8afe3 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:24.081098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23950b88-d55f-40bd-ac27-08148ed8afe3 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:24.088310 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:24 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:24.097865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1786781-20dd-46a2-98bd-cc88ffdf41e9 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:24.098277 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1786781-20dd-46a2-98bd-cc88ffdf41e9 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:24.098457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1786781-20dd-46a2-98bd-cc88ffdf41e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:24.178031 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: INFO dbcounter [None req-a1786781-20dd-46a2-98bd-cc88ffdf41e9 None None] Registered counter for database keystone Aug 23 06:58:24.199492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG oslo_db.sqlalchemy.engines [None req-a1786781-20dd-46a2-98bd-cc88ffdf41e9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81714) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 23 06:58:24.211633 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writer thread running {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 23 06:58:24.398459 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG passlib.handlers.bcrypt [None req-a1786781-20dd-46a2-98bd-cc88ffdf41e9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81714) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 23 06:58:24.398856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG passlib.handlers.bcrypt [None req-a1786781-20dd-46a2-98bd-cc88ffdf41e9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81714) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 23 06:58:24.454571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-a1786781-20dd-46a2-98bd-cc88ffdf41e9 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:24.456241 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-a1786781-20dd-46a2-98bd-cc88ffdf41e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:24.602281 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 506 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 23 06:58:25.104893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23950b88-d55f-40bd-ac27-08148ed8afe3 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:25.175010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: INFO dbcounter [None req-23950b88-d55f-40bd-ac27-08148ed8afe3 None None] Registered counter for database keystone Aug 23 06:58:25.200247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG oslo_db.sqlalchemy.engines [None req-23950b88-d55f-40bd-ac27-08148ed8afe3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81715) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 23 06:58:25.209740 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writer thread running {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 23 06:58:25.337258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23950b88-d55f-40bd-ac27-08148ed8afe3 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:25.342717 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cb925-95d3-4879-a194-3a807ad428fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:25.343356 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cb925-95d3-4879-a194-3a807ad428fb admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:25.343684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cb925-95d3-4879-a194-3a807ad428fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:25.343955 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cb925-95d3-4879-a194-3a807ad428fb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:25.353140 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cb925-95d3-4879-a194-3a807ad428fb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:25.405452 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e31cb925-95d3-4879-a194-3a807ad428fb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 23 06:58:25.405805 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cb925-95d3-4879-a194-3a807ad428fb admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:25.416635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Tue Aug 23 06:58:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 06:58:25.423523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1786781-20dd-46a2-98bd-cc88ffdf41e9 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:25.439142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1786781-20dd-46a2-98bd-cc88ffdf41e9 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:25.444531 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcc4d1-c150-47d3-b6c8-0c7f8238e641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:25.445634 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcc4d1-c150-47d3-b6c8-0c7f8238e641 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:25.445924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcc4d1-c150-47d3-b6c8-0c7f8238e641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:25.446190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcc4d1-c150-47d3-b6c8-0c7f8238e641 admin admin] PATH_INFO: `/v3/projects/c7014afbcdaa44f68b1b75e344808f41` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:25.447826 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bcc4d1-c150-47d3-b6c8-0c7f8238e641 admin admin] RBAC: Authorizing `identity:get_project(project_id=c7014afbcdaa44f68b1b75e344808f41)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:25.500814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: WARNING keystone.common.rbac_enforcer.enforcer [None req-44bcc4d1-c150-47d3-b6c8-0c7f8238e641 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 23 06:58:25.501155 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bcc4d1-c150-47d3-b6c8-0c7f8238e641 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:25.505752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Tue Aug 23 06:58:25 2022] GET /identity/v3/projects/c7014afbcdaa44f68b1b75e344808f41 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:58:27.393047 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584f672-b9d0-4b5b-9f57-866d67e6c497 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:27.393914 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584f672-b9d0-4b5b-9f57-866d67e6c497 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:27.393914 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584f672-b9d0-4b5b-9f57-866d67e6c497 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:27.400701 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:27.407285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9b215-9a14-4d27-8e85-e69ab53af333 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:27.407775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9b215-9a14-4d27-8e85-e69ab53af333 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:27.407869 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9b215-9a14-4d27-8e85-e69ab53af333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:27.433967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-52e9b215-9a14-4d27-8e85-e69ab53af333 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:27.435550 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-52e9b215-9a14-4d27-8e85-e69ab53af333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:27.444923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 23 06:58:27.712885 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584f672-b9d0-4b5b-9f57-866d67e6c497 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:27.729687 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584f672-b9d0-4b5b-9f57-866d67e6c497 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:27.735151 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cad08-6435-4129-8087-495bdcfa3363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:27.735791 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cad08-6435-4129-8087-495bdcfa3363 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:27.736070 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cad08-6435-4129-8087-495bdcfa3363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:27.736344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cad08-6435-4129-8087-495bdcfa3363 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:27.745330 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cad08-6435-4129-8087-495bdcfa3363 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:27.747149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cad08-6435-4129-8087-495bdcfa3363 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:27.756685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Tue Aug 23 06:58:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 06:58:27.765358 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9b215-9a14-4d27-8e85-e69ab53af333 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:27.780140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9b215-9a14-4d27-8e85-e69ab53af333 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:27.786446 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf817b8-bb3a-4a7a-9d37-50cece11bf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:27.787074 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf817b8-bb3a-4a7a-9d37-50cece11bf13 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:27.787350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf817b8-bb3a-4a7a-9d37-50cece11bf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:27.787650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf817b8-bb3a-4a7a-9d37-50cece11bf13 admin admin] PATH_INFO: `/v3/users/2ff4f07bfac14f428ece0dfbc5ec3579` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:27.789116 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf817b8-bb3a-4a7a-9d37-50cece11bf13 admin admin] RBAC: Authorizing `identity:get_user(user_id=2ff4f07bfac14f428ece0dfbc5ec3579)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:27.790983 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf817b8-bb3a-4a7a-9d37-50cece11bf13 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:27.792562 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Tue Aug 23 06:58:27 2022] GET /identity/v3/users/2ff4f07bfac14f428ece0dfbc5ec3579 => generated 247 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:29.740344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45454baf-522c-44d1-be2f-fae17b1f9624 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:29.740871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45454baf-522c-44d1-be2f-fae17b1f9624 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:29.740871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45454baf-522c-44d1-be2f-fae17b1f9624 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:29.742184 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:29.747601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831129a-09c1-4c15-ac57-f418e2a016e1 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:29.748030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831129a-09c1-4c15-ac57-f418e2a016e1 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:29.748125 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831129a-09c1-4c15-ac57-f418e2a016e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:29.766467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-9831129a-09c1-4c15-ac57-f418e2a016e1 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:29.767845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9831129a-09c1-4c15-ac57-f418e2a016e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:29.792497 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825430d-0e1b-4157-aaa3-95fc2919fc5d None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:29.792777 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825430d-0e1b-4157-aaa3-95fc2919fc5d None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:29.793046 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825430d-0e1b-4157-aaa3-95fc2919fc5d None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:29.794350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:29.799712 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2522d-3499-4ec2-a17e-b68c3543f16d None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:29.799994 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2522d-3499-4ec2-a17e-b68c3543f16d None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:29.800348 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2522d-3499-4ec2-a17e-b68c3543f16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:29.844561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG passlib.handlers.bcrypt [None req-d3f2522d-3499-4ec2-a17e-b68c3543f16d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81715) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 23 06:58:29.844978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG passlib.handlers.bcrypt [None req-d3f2522d-3499-4ec2-a17e-b68c3543f16d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81715) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 23 06:58:29.850529 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:29.890059 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-d3f2522d-3499-4ec2-a17e-b68c3543f16d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:29.891680 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d3f2522d-3499-4ec2-a17e-b68c3543f16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:29.993191 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:30.113672 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831129a-09c1-4c15-ac57-f418e2a016e1 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:30.140332 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:30.225072 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831129a-09c1-4c15-ac57-f418e2a016e1 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:30.253821 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2522d-3499-4ec2-a17e-b68c3543f16d None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:30.259655 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f66fe9-d5ea-417d-bee5-b1582a5fab8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:30.260309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f66fe9-d5ea-417d-bee5-b1582a5fab8e None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:30.260587 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f66fe9-d5ea-417d-bee5-b1582a5fab8e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:30.260848 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f66fe9-d5ea-417d-bee5-b1582a5fab8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:30.272604 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f66fe9-d5ea-417d-bee5-b1582a5fab8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:30.281309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-31f66fe9-d5ea-417d-bee5-b1582a5fab8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:58:30.281309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:30.281309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:30.281309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:30.281309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:30.281309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:30.281309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:30.281309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:30.281309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:30.281309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:30.281309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:30.281309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:30.281309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:30.281309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:30.281309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:30.281309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:30.282214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:30.282214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:30.282214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:30.282214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:30.282214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:30.282214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:30.282214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:30.282214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:30.282214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:30.282214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:30.282214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:30.282214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:30.282214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:30.282214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:30.282214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:30.282214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:30.283141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:30.283141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:30.283141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:30.283141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:30.283141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:30.283141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:30.283141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:30.283141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:30.283141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:30.283141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:30.283141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:58:30.283141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:58:30.283141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 06:58:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:58:30.288351 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f66fe9-d5ea-417d-bee5-b1582a5fab8e None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:30.357285 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2522d-3499-4ec2-a17e-b68c3543f16d None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:30.359592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:30.371881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:30.392045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f66fe9-d5ea-417d-bee5-b1582a5fab8e None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:30.396241 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75e87f-0069-4f6c-9b25-2d8a5b1e30bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:30.397101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75e87f-0069-4f6c-9b25-2d8a5b1e30bc None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:30.397429 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75e87f-0069-4f6c-9b25-2d8a5b1e30bc None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:30.397701 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75e87f-0069-4f6c-9b25-2d8a5b1e30bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:30.436113 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b3002-fd7e-4525-812f-7f69d06f6082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:30.436731 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b3002-fd7e-4525-812f-7f69d06f6082 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:30.437011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b3002-fd7e-4525-812f-7f69d06f6082 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:30.437279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b3002-fd7e-4525-812f-7f69d06f6082 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:30.437957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b3002-fd7e-4525-812f-7f69d06f6082 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:30.438258 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b3002-fd7e-4525-812f-7f69d06f6082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:30.439798 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b3002-fd7e-4525-812f-7f69d06f6082 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:30.454048 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:58:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:58:30.459991 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b3002-fd7e-4525-812f-7f69d06f6082 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:30.482333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75e87f-0069-4f6c-9b25-2d8a5b1e30bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:30.484623 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75e87f-0069-4f6c-9b25-2d8a5b1e30bc None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:30.560726 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b3002-fd7e-4525-812f-7f69d06f6082 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:30.582270 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 8/16] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:58:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 06:58:30.588187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75e87f-0069-4f6c-9b25-2d8a5b1e30bc None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:30.591493 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:30.595225 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f4b6e-a06d-4902-84bd-a7e62008f817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:30.595904 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f4b6e-a06d-4902-84bd-a7e62008f817 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:30.596316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f4b6e-a06d-4902-84bd-a7e62008f817 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:30.596596 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f4b6e-a06d-4902-84bd-a7e62008f817 None admin] PATH_INFO: `/v3/users/2ff4f07bfac14f428ece0dfbc5ec3579` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:30.597755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f4b6e-a06d-4902-84bd-a7e62008f817 None admin] RBAC: Authorizing `identity:get_user(user_id=2ff4f07bfac14f428ece0dfbc5ec3579)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:30.599237 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f4b6e-a06d-4902-84bd-a7e62008f817 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:30.600870 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:58:30 2022] GET /identity/v3/users/2ff4f07bfac14f428ece0dfbc5ec3579 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:30.606410 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f4b6e-a06d-4902-84bd-a7e62008f817 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:30.621781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:30.695000 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75e87f-0069-4f6c-9b25-2d8a5b1e30bc None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:30.709954 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f4b6e-a06d-4902-84bd-a7e62008f817 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:30.730418 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633e2ee-c3be-4c05-87c0-f18aa3ab654d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:30.731284 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633e2ee-c3be-4c05-87c0-f18aa3ab654d None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:30.731593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633e2ee-c3be-4c05-87c0-f18aa3ab654d None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:30.731869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633e2ee-c3be-4c05-87c0-f18aa3ab654d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:30.743118 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c7300-989d-4805-9f21-3fd41a423683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:30.743777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c7300-989d-4805-9f21-3fd41a423683 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:30.744055 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c7300-989d-4805-9f21-3fd41a423683 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:30.744815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c7300-989d-4805-9f21-3fd41a423683 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:30.745979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c7300-989d-4805-9f21-3fd41a423683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:30.747489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c7300-989d-4805-9f21-3fd41a423683 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:30.749565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 10/18] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:58:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:58:30.752862 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633e2ee-c3be-4c05-87c0-f18aa3ab654d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:30.754355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633e2ee-c3be-4c05-87c0-f18aa3ab654d None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:30.757182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c7300-989d-4805-9f21-3fd41a423683 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:30.772776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-d633e2ee-c3be-4c05-87c0-f18aa3ab654d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:58:30.772776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:30.772776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:30.772776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:30.772776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:30.772776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:30.772776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:30.772776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:30.772776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:30.772776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:30.772776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:30.772776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:30.772776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:58:30.772776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:58:30.772776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:58:30.772776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:58:30.773666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:30.773666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:30.773666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:30.773666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:30.773666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:30.773666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:30.773666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:30.773666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:30.773666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:30.773666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:30.773666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:30.773666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:30.773666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:30.773666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:30.773666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:30.773666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:30.773666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:58:30.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:58:30.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:58:30.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:58:30.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 9/19] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:58:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:58:30.779878 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633e2ee-c3be-4c05-87c0-f18aa3ab654d None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:30.838349 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:30.840688 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:30.859521 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c7300-989d-4805-9f21-3fd41a423683 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:30.884556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633e2ee-c3be-4c05-87c0-f18aa3ab654d None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:30.893917 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c291e03-d495-4dbc-9c4a-765fae00a781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:30.894535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c291e03-d495-4dbc-9c4a-765fae00a781 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:30.894824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c291e03-d495-4dbc-9c4a-765fae00a781 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:30.895097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c291e03-d495-4dbc-9c4a-765fae00a781 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:30.895865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c291e03-d495-4dbc-9c4a-765fae00a781 None admin] RBAC: Adding query filter params (role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.domain.id=default, user.id=2ff4f07bfac14f428ece0dfbc5ec3579) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:30.896176 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c291e03-d495-4dbc-9c4a-765fae00a781 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:30.897929 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c291e03-d495-4dbc-9c4a-765fae00a781 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:30.904603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 11/20] 10.222.0.17 () {56 vars in 1322 bytes} [Tue Aug 23 06:58:30 2022] GET /identity/v3/role_assignments?user.id=2ff4f07bfac14f428ece0dfbc5ec3579&scope.domain.id=default&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:58:30.918804 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec6146-013d-4bd9-9cc3-4547ff6685db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:30.919469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec6146-013d-4bd9-9cc3-4547ff6685db None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:30.919784 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec6146-013d-4bd9-9cc3-4547ff6685db None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:30.920045 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec6146-013d-4bd9-9cc3-4547ff6685db None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:30.920721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec6146-013d-4bd9-9cc3-4547ff6685db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:30.921025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec6146-013d-4bd9-9cc3-4547ff6685db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:30.922486 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec6146-013d-4bd9-9cc3-4547ff6685db None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:30.945537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 10/21] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:33.033547 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898f21d-8977-4992-893b-d9f29c364365 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:33.034114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898f21d-8977-4992-893b-d9f29c364365 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:33.034114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898f21d-8977-4992-893b-d9f29c364365 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:33.035438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 12/22] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:33.041006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e6178-606a-4426-8ef7-d1f27e2384c4 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:33.041831 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e6178-606a-4426-8ef7-d1f27e2384c4 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:33.042279 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e6178-606a-4426-8ef7-d1f27e2384c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:33.084519 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-335e6178-606a-4426-8ef7-d1f27e2384c4 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:33.086035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-335e6178-606a-4426-8ef7-d1f27e2384c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:33.241354 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 11/23] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:58:33.435011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d64fc1-879c-4569-94fd-3443929758d0 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:33.435382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d64fc1-879c-4569-94fd-3443929758d0 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:33.435617 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d64fc1-879c-4569-94fd-3443929758d0 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:33.437044 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 13/24] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:33.443680 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee316a8c-4e12-4eff-901b-95b35b516c7e None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:33.444172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee316a8c-4e12-4eff-901b-95b35b516c7e None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:33.444663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee316a8c-4e12-4eff-901b-95b35b516c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:33.481349 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-ee316a8c-4e12-4eff-901b-95b35b516c7e None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:33.482550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ee316a8c-4e12-4eff-901b-95b35b516c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:33.490273 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68e64a-2d9b-4e79-924a-d719f71bd718 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:33.490555 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68e64a-2d9b-4e79-924a-d719f71bd718 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:33.490831 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68e64a-2d9b-4e79-924a-d719f71bd718 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:33.492384 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 14/25] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:58:33.497456 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab394f9-5e6b-4d7f-b964-a4d7b682000b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:33.497739 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab394f9-5e6b-4d7f-b964-a4d7b682000b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:33.498069 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab394f9-5e6b-4d7f-b964-a4d7b682000b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:33.528429 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-fab394f9-5e6b-4d7f-b964-a4d7b682000b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:33.529645 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-fab394f9-5e6b-4d7f-b964-a4d7b682000b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:33.568099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:33.612104 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 12/26] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:58:33.613437 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5d8c1-5c89-4b5f-99e8-6c3e7ecf3ac1 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:33.613721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5d8c1-5c89-4b5f-99e8-6c3e7ecf3ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:33.613989 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5d8c1-5c89-4b5f-99e8-6c3e7ecf3ac1 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:33.617537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 13/27] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:33.624795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e91075-8cf1-44b8-9666-2cdb07882afe None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:33.625080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e91075-8cf1-44b8-9666-2cdb07882afe None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:33.625351 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e91075-8cf1-44b8-9666-2cdb07882afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:33.648804 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-e0e91075-8cf1-44b8-9666-2cdb07882afe None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:33.649989 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e0e91075-8cf1-44b8-9666-2cdb07882afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:33.653472 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 15/28] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:33.710953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab394f9-5e6b-4d7f-b964-a4d7b682000b None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:33.738994 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 14/29] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:33.803057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:33.837107 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab394f9-5e6b-4d7f-b964-a4d7b682000b None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:33.882843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc88dc8-d980-4987-8d38-b68cbc805e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:33.883520 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc88dc8-d980-4987-8d38-b68cbc805e6d None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:33.883807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc88dc8-d980-4987-8d38-b68cbc805e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:33.884084 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc88dc8-d980-4987-8d38-b68cbc805e6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:33.894844 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc88dc8-d980-4987-8d38-b68cbc805e6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:33.899752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-2bc88dc8-d980-4987-8d38-b68cbc805e6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:58:33.899752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:33.899752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:33.899752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:33.899752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:33.899752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:33.899752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:33.899752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:33.899752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:33.899752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:33.899752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:33.899752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:33.899752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:33.899752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:33.899752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:33.899752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:33.900579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:33.900579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:33.900579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:33.900579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:33.900579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:33.900579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:33.900579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:33.900579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:33.900579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:33.900579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:33.900579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:33.900579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:33.900579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:33.900579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:33.900579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:33.900579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:33.901337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:33.901337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:33.901337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:33.901337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:33.901337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:33.901337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:33.901337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:33.901337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:33.901337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:33.901337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:33.901337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:58:33.901337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:58:33.901337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 16/30] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 06:58:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:58:33.906834 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e91075-8cf1-44b8-9666-2cdb07882afe None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:34.021080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e91075-8cf1-44b8-9666-2cdb07882afe None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:34.026476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc88dc8-d980-4987-8d38-b68cbc805e6d None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:34.029494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:34.074817 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90e710-7f23-4cbc-8535-b7879ca17cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:34.074817 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90e710-7f23-4cbc-8535-b7879ca17cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:34.074817 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90e710-7f23-4cbc-8535-b7879ca17cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:34.074817 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90e710-7f23-4cbc-8535-b7879ca17cc7 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:34.075828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90e710-7f23-4cbc-8535-b7879ca17cc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:34.075828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90e710-7f23-4cbc-8535-b7879ca17cc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:34.075828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90e710-7f23-4cbc-8535-b7879ca17cc7 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:34.090199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 15/31] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:58:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:34.091052 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90e710-7f23-4cbc-8535-b7879ca17cc7 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:34.210559 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc88dc8-d980-4987-8d38-b68cbc805e6d None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:34.249853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90e710-7f23-4cbc-8535-b7879ca17cc7 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:34.251102 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de217d-0811-4c5f-a03d-77e5aa904809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:34.251900 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de217d-0811-4c5f-a03d-77e5aa904809 None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:34.252178 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de217d-0811-4c5f-a03d-77e5aa904809 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:34.253237 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de217d-0811-4c5f-a03d-77e5aa904809 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:34.253925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de217d-0811-4c5f-a03d-77e5aa904809 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:34.255336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de217d-0811-4c5f-a03d-77e5aa904809 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:34.259275 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:34.292041 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828dd018-7bc1-42c7-a23f-8c77a5f84aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:34.292761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828dd018-7bc1-42c7-a23f-8c77a5f84aaa None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:34.293115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828dd018-7bc1-42c7-a23f-8c77a5f84aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:34.293495 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828dd018-7bc1-42c7-a23f-8c77a5f84aaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:34.304728 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828dd018-7bc1-42c7-a23f-8c77a5f84aaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:34.306377 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828dd018-7bc1-42c7-a23f-8c77a5f84aaa None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:34.322918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-828dd018-7bc1-42c7-a23f-8c77a5f84aaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:58:34.322918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:34.322918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:34.322918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:34.322918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:34.322918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:34.322918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:34.322918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:34.322918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:34.322918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:34.322918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:34.322918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:34.322918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:58:34.322918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:58:34.322918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:58:34.322918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:58:34.323838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:34.323838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:34.323838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:34.323838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:34.323838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:34.323838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:34.323838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:34.323838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:34.323838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:34.323838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:34.323838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:34.323838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:34.323838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:34.323838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:34.323838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:34.323838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:34.323838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:58:34.325199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:58:34.325199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:58:34.325199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:58:34.325199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 16/32] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:58:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:58:34.330060 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828dd018-7bc1-42c7-a23f-8c77a5f84aaa None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:34.355113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:34.403255 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 17/33] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Aug 23 06:58:34 2022] POST /identity/v3/roles => generated 219 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 06:58:34.411479 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de217d-0811-4c5f-a03d-77e5aa904809 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:34.476138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828dd018-7bc1-42c7-a23f-8c77a5f84aaa None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:34.515550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68f583-f407-4937-99d5-11a81a93faca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:34.516435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68f583-f407-4937-99d5-11a81a93faca None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:34.516848 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68f583-f407-4937-99d5-11a81a93faca None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:34.517255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68f583-f407-4937-99d5-11a81a93faca None admin] PATH_INFO: `/v3/users/2ff4f07bfac14f428ece0dfbc5ec3579` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:34.519332 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68f583-f407-4937-99d5-11a81a93faca None admin] RBAC: Authorizing `identity:get_user(user_id=2ff4f07bfac14f428ece0dfbc5ec3579)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:34.521122 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68f583-f407-4937-99d5-11a81a93faca None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:34.523376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 17/34] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:58:34 2022] GET /identity/v3/users/2ff4f07bfac14f428ece0dfbc5ec3579 => generated 247 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:34.524394 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68f583-f407-4937-99d5-11a81a93faca None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:34.623161 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:34.632444 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de217d-0811-4c5f-a03d-77e5aa904809 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:34.638925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68f583-f407-4937-99d5-11a81a93faca None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:34.677299 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae1df0-e096-45ca-8730-411e748bf72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:34.678027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae1df0-e096-45ca-8730-411e748bf72e None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:34.678325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae1df0-e096-45ca-8730-411e748bf72e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:34.678603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae1df0-e096-45ca-8730-411e748bf72e None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:34.679390 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae1df0-e096-45ca-8730-411e748bf72e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:34.684819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae1df0-e096-45ca-8730-411e748bf72e None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:34.690300 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c91ce-b989-41ca-ada7-a8f8b038422b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:34.691189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c91ce-b989-41ca-ada7-a8f8b038422b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:34.691586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c91ce-b989-41ca-ada7-a8f8b038422b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:34.691888 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c91ce-b989-41ca-ada7-a8f8b038422b None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:34.692723 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c91ce-b989-41ca-ada7-a8f8b038422b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:34.693038 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c91ce-b989-41ca-ada7-a8f8b038422b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:34.694612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c91ce-b989-41ca-ada7-a8f8b038422b None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:34.704696 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 18/35] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Aug 23 06:58:34 2022] POST /identity/v3/roles => generated 225 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:58:34.706952 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae1df0-e096-45ca-8730-411e748bf72e None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:34.709178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 18/36] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:34.715132 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c91ce-b989-41ca-ada7-a8f8b038422b None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:34.795349 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:34.856720 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae1df0-e096-45ca-8730-411e748bf72e None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:34.897204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c91ce-b989-41ca-ada7-a8f8b038422b None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:34.924723 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550431c9-0116-4872-b67b-7e724453c9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:34.925571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550431c9-0116-4872-b67b-7e724453c9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:34.925870 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550431c9-0116-4872-b67b-7e724453c9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:34.926320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550431c9-0116-4872-b67b-7e724453c9d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:34.927799 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550431c9-0116-4872-b67b-7e724453c9d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:34.929306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550431c9-0116-4872-b67b-7e724453c9d2 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:34.931106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 19/37] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:58:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:34.937978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550431c9-0116-4872-b67b-7e724453c9d2 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:34.939391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a80260-4c29-469f-9ab0-777eef0fc4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:34.940260 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a80260-4c29-469f-9ab0-777eef0fc4ae None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:34.940787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a80260-4c29-469f-9ab0-777eef0fc4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:34.941458 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a80260-4c29-469f-9ab0-777eef0fc4ae None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:34.943238 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a80260-4c29-469f-9ab0-777eef0fc4ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:34.944998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a80260-4c29-469f-9ab0-777eef0fc4ae None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:34.983344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:35.066541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550431c9-0116-4872-b67b-7e724453c9d2 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:35.090119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:35.098282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 19/38] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 23 06:58:34 2022] POST /identity/v3/projects => generated 301 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 06:58:35.101525 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027defba-8f6e-4212-8c96-1289e847aadb None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:35.101995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027defba-8f6e-4212-8c96-1289e847aadb None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:35.102409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027defba-8f6e-4212-8c96-1289e847aadb None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:35.104221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 20/39] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:35.109977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db335889-5bba-4215-a096-74c73f7cbae8 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:35.110496 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db335889-5bba-4215-a096-74c73f7cbae8 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:35.111206 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db335889-5bba-4215-a096-74c73f7cbae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:35.114408 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbc2e0-a17d-4c84-b9ac-d40124a77a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:35.115118 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbc2e0-a17d-4c84-b9ac-d40124a77a4e None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:35.115403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbc2e0-a17d-4c84-b9ac-d40124a77a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:35.115726 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbc2e0-a17d-4c84-b9ac-d40124a77a4e None admin] PATH_INFO: `/v3/domains/default/users/2ff4f07bfac14f428ece0dfbc5ec3579/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:35.117765 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbc2e0-a17d-4c84-b9ac-d40124a77a4e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2ff4f07bfac14f428ece0dfbc5ec3579, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:35.119301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbc2e0-a17d-4c84-b9ac-d40124a77a4e None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:35.134845 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-db335889-5bba-4215-a096-74c73f7cbae8 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:35.136457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 20/40] 10.222.0.17 () {58 vars in 1361 bytes} [Tue Aug 23 06:58:34 2022] PUT /identity/v3/domains/default/users/2ff4f07bfac14f428ece0dfbc5ec3579/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 06:58:35.136589 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-db335889-5bba-4215-a096-74c73f7cbae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:35.137915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18585b32-9255-4077-b9b0-b24cc9a7fe68 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:35.138276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18585b32-9255-4077-b9b0-b24cc9a7fe68 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:35.138573 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18585b32-9255-4077-b9b0-b24cc9a7fe68 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:35.140375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 21/41] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:35.145905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8688ebf-6a93-4e3e-802a-f8b857d752f8 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:35.146211 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8688ebf-6a93-4e3e-802a-f8b857d752f8 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:35.146487 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8688ebf-6a93-4e3e-802a-f8b857d752f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:35.166100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-e8688ebf-6a93-4e3e-802a-f8b857d752f8 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:35.167743 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e8688ebf-6a93-4e3e-802a-f8b857d752f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:35.228087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 21/42] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:35.260034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 22/43] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:58:35.486925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db335889-5bba-4215-a096-74c73f7cbae8 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:35.546741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8688ebf-6a93-4e3e-802a-f8b857d752f8 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:35.590324 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db335889-5bba-4215-a096-74c73f7cbae8 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:35.620016 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:35.626510 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1e298-3a1c-401c-a311-ade9ebc47be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:35.627215 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1e298-3a1c-401c-a311-ade9ebc47be5 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:35.627587 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1e298-3a1c-401c-a311-ade9ebc47be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:35.627866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1e298-3a1c-401c-a311-ade9ebc47be5 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:35.628596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d1e298-3a1c-401c-a311-ade9ebc47be5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:35.630342 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d1e298-3a1c-401c-a311-ade9ebc47be5 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:35.648059 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 22/44] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Aug 23 06:58:35 2022] POST /identity/v3/roles => generated 223 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:58:35.659829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8688ebf-6a93-4e3e-802a-f8b857d752f8 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:35.676428 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:35.707268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83805a07-c07a-4573-92a0-4039599380f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:35.707951 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83805a07-c07a-4573-92a0-4039599380f5 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:35.708233 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83805a07-c07a-4573-92a0-4039599380f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:35.708506 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83805a07-c07a-4573-92a0-4039599380f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:35.709735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83805a07-c07a-4573-92a0-4039599380f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:35.711237 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83805a07-c07a-4573-92a0-4039599380f5 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:35.713274 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 23/45] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:58:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:35.718793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1e298-3a1c-401c-a311-ade9ebc47be5 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:35.835808 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1e298-3a1c-401c-a311-ade9ebc47be5 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:35.870098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77f512-0c98-4b15-a8ee-51bce4290105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:35.870720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77f512-0c98-4b15-a8ee-51bce4290105 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:35.871005 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77f512-0c98-4b15-a8ee-51bce4290105 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:35.871281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77f512-0c98-4b15-a8ee-51bce4290105 None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:35.871968 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77f512-0c98-4b15-a8ee-51bce4290105 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:35.873452 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77f512-0c98-4b15-a8ee-51bce4290105 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:35.900658 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 23/46] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 23 06:58:35 2022] POST /identity/v3/projects => generated 312 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:58:37.075517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681f845-b3eb-4cbd-95c7-8c88ebf931d5 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:37.076020 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681f845-b3eb-4cbd-95c7-8c88ebf931d5 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:37.076020 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681f845-b3eb-4cbd-95c7-8c88ebf931d5 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:37.077337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 24/47] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:37.082842 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d7674-b8be-4175-aa7a-ba2db270847f None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:37.083245 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d7674-b8be-4175-aa7a-ba2db270847f None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:37.083372 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d7674-b8be-4175-aa7a-ba2db270847f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:37.102241 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-c59d7674-b8be-4175-aa7a-ba2db270847f None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:37.103443 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c59d7674-b8be-4175-aa7a-ba2db270847f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:37.141433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:37.174604 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 24/48] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:37.438446 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681f845-b3eb-4cbd-95c7-8c88ebf931d5 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:37.544786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681f845-b3eb-4cbd-95c7-8c88ebf931d5 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:37.692709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdc5d3-0ba9-4075-9f73-b3f7e7d42b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:37.693288 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdc5d3-0ba9-4075-9f73-b3f7e7d42b51 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:37.693572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdc5d3-0ba9-4075-9f73-b3f7e7d42b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:37.693857 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdc5d3-0ba9-4075-9f73-b3f7e7d42b51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:37.704690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fdc5d3-0ba9-4075-9f73-b3f7e7d42b51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:37.709852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-f3fdc5d3-0ba9-4075-9f73-b3f7e7d42b51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:58:37.709852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:37.709852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:37.709852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:37.709852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:37.709852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:37.709852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:37.709852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:37.709852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:37.709852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:37.709852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:37.709852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:37.709852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:37.709852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:37.709852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:37.709852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:37.710699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:37.710699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:37.710699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:37.710699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:37.710699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:37.710699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:37.710699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:37.710699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:37.710699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:37.710699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:37.710699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:37.710699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:37.710699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:37.710699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:37.710699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:37.710699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:37.711750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:37.711750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:37.711750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:37.711750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:37.711750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:37.711750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:37.711750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:37.711750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:37.711750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:37.711750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:37.711750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:58:37.711750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:58:37.711750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 25/49] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 06:58:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:58:37.716956 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d7674-b8be-4175-aa7a-ba2db270847f None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:37.773739 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1612b-fa4c-405b-bccc-2c87335b968a None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:37.773975 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1612b-fa4c-405b-bccc-2c87335b968a None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:37.774267 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1612b-fa4c-405b-bccc-2c87335b968a None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:37.775652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 26/50] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:58:37.783373 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ad085-976b-4831-9bc8-7def04d39878 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:37.783854 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ad085-976b-4831-9bc8-7def04d39878 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:37.784266 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ad085-976b-4831-9bc8-7def04d39878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:37.803396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-878ad085-976b-4831-9bc8-7def04d39878 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:37.804650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-878ad085-976b-4831-9bc8-7def04d39878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:37.819837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d7674-b8be-4175-aa7a-ba2db270847f None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:37.840386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:37.854904 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb2588-53f8-4a0b-b765-acc689229074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:37.855558 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb2588-53f8-4a0b-b765-acc689229074 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:37.855841 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb2588-53f8-4a0b-b765-acc689229074 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:37.856117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb2588-53f8-4a0b-b765-acc689229074 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:37.856789 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb2588-53f8-4a0b-b765-acc689229074 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:37.857093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb2588-53f8-4a0b-b765-acc689229074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:37.858499 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb2588-53f8-4a0b-b765-acc689229074 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:37.866705 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 25/51] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:58:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:37.872273 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb2588-53f8-4a0b-b765-acc689229074 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:37.874106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 27/52] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:37.909137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:37.975683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb2588-53f8-4a0b-b765-acc689229074 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:38.010669 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da1621-694f-48fc-8b20-ff989da7f97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:38.011318 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da1621-694f-48fc-8b20-ff989da7f97d None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:38.011632 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da1621-694f-48fc-8b20-ff989da7f97d None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:38.011908 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da1621-694f-48fc-8b20-ff989da7f97d None admin] PATH_INFO: `/v3/users/2ff4f07bfac14f428ece0dfbc5ec3579` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:38.013111 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da1621-694f-48fc-8b20-ff989da7f97d None admin] RBAC: Authorizing `identity:get_user(user_id=2ff4f07bfac14f428ece0dfbc5ec3579)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:38.014564 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da1621-694f-48fc-8b20-ff989da7f97d None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:38.016173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 26/53] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:58:37 2022] GET /identity/v3/users/2ff4f07bfac14f428ece0dfbc5ec3579 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:38.022259 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ad085-976b-4831-9bc8-7def04d39878 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:38.125202 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ad085-976b-4831-9bc8-7def04d39878 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:38.149271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da1621-694f-48fc-8b20-ff989da7f97d None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:38.159463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44ad1d-50d0-47ee-b877-fcd9bda7f8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:38.160106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44ad1d-50d0-47ee-b877-fcd9bda7f8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:38.160429 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44ad1d-50d0-47ee-b877-fcd9bda7f8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:38.160735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44ad1d-50d0-47ee-b877-fcd9bda7f8f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:38.162053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44ad1d-50d0-47ee-b877-fcd9bda7f8f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:38.163592 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44ad1d-50d0-47ee-b877-fcd9bda7f8f2 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:38.165173 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 28/54] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:58:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:38.171031 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44ad1d-50d0-47ee-b877-fcd9bda7f8f2 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:38.213202 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:38.248467 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da1621-694f-48fc-8b20-ff989da7f97d None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:38.249949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:38.274860 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44ad1d-50d0-47ee-b877-fcd9bda7f8f2 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:38.283731 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc30c49-3312-47c1-8029-86ba1bd56a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:38.284733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc30c49-3312-47c1-8029-86ba1bd56a7a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:38.285019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc30c49-3312-47c1-8029-86ba1bd56a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:38.285623 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc30c49-3312-47c1-8029-86ba1bd56a7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:38.286761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc30c49-3312-47c1-8029-86ba1bd56a7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:38.288261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc30c49-3312-47c1-8029-86ba1bd56a7a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:38.289854 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 27/55] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:58:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:38.295464 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc30c49-3312-47c1-8029-86ba1bd56a7a None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:38.308313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f9b2f-3b30-4971-a7c3-268d43e72e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:38.308928 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f9b2f-3b30-4971-a7c3-268d43e72e7d None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:38.309211 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f9b2f-3b30-4971-a7c3-268d43e72e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:38.309489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f9b2f-3b30-4971-a7c3-268d43e72e7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:38.310544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f9b2f-3b30-4971-a7c3-268d43e72e7d None admin] RBAC: Adding query filter params (role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.domain.id=default, user.id=2ff4f07bfac14f428ece0dfbc5ec3579) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:38.310851 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f9b2f-3b30-4971-a7c3-268d43e72e7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:38.312352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f9b2f-3b30-4971-a7c3-268d43e72e7d None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:38.318498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 29/56] 10.222.0.17 () {56 vars in 1322 bytes} [Tue Aug 23 06:58:38 2022] GET /identity/v3/role_assignments?user.id=2ff4f07bfac14f428ece0dfbc5ec3579&scope.domain.id=default&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:38.403546 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc30c49-3312-47c1-8029-86ba1bd56a7a None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:38.442721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbe421-859f-48db-b349-6e1cd353ccf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:38.443396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbe421-859f-48db-b349-6e1cd353ccf4 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:38.443706 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbe421-859f-48db-b349-6e1cd353ccf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:38.443984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbe421-859f-48db-b349-6e1cd353ccf4 None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:38.444648 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbe421-859f-48db-b349-6e1cd353ccf4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:38.446123 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbe421-859f-48db-b349-6e1cd353ccf4 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:38.478206 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 28/57] 10.222.0.17 () {60 vars in 1140 bytes} [Tue Aug 23 06:58:38 2022] POST /identity/v3/projects => generated 298 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 06:58:40.418907 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbe051-d642-4ecb-9650-19f14a1ba4ff None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:40.419779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbe051-d642-4ecb-9650-19f14a1ba4ff None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:40.419914 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbe051-d642-4ecb-9650-19f14a1ba4ff None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:40.421956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 30/58] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:40.428294 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5b6a0-c050-4c2a-9369-9ce02328e590 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:40.429002 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5b6a0-c050-4c2a-9369-9ce02328e590 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:40.429122 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5b6a0-c050-4c2a-9369-9ce02328e590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:40.453190 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-b4c5b6a0-c050-4c2a-9369-9ce02328e590 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:40.454400 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b4c5b6a0-c050-4c2a-9369-9ce02328e590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:40.471561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:40.532401 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 29/59] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:40.797218 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbe051-d642-4ecb-9650-19f14a1ba4ff None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:40.923932 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbe051-d642-4ecb-9650-19f14a1ba4ff None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:40.926375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:40.961433 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3248c33-4c60-4012-ad42-75c990c1ddc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:40.962058 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3248c33-4c60-4012-ad42-75c990c1ddc0 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:40.962335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3248c33-4c60-4012-ad42-75c990c1ddc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:40.962608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3248c33-4c60-4012-ad42-75c990c1ddc0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:40.963781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3248c33-4c60-4012-ad42-75c990c1ddc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:40.965247 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3248c33-4c60-4012-ad42-75c990c1ddc0 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:40.966828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 31/60] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:58:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:40.972616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5b6a0-c050-4c2a-9369-9ce02328e590 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:41.080997 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5b6a0-c050-4c2a-9369-9ce02328e590 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:41.120620 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56dc8ba-12d8-4996-bb66-11ad5d3ed6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:41.121228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dc8ba-12d8-4996-bb66-11ad5d3ed6d7 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:41.121506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dc8ba-12d8-4996-bb66-11ad5d3ed6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:41.121784 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dc8ba-12d8-4996-bb66-11ad5d3ed6d7 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:41.122431 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56dc8ba-12d8-4996-bb66-11ad5d3ed6d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:41.123934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56dc8ba-12d8-4996-bb66-11ad5d3ed6d7 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:41.152825 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 30/61] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 23 06:58:40 2022] POST /identity/v3/users => generated 275 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:58:44.117936 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5d17d-b8e1-4582-9e05-d1916313b8c3 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:44.118425 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5d17d-b8e1-4582-9e05-d1916313b8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:44.118425 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5d17d-b8e1-4582-9e05-d1916313b8c3 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:44.119789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 32/62] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:44.133318 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a200d0d-6d71-4418-baa2-7101c536466b None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:44.133814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a200d0d-6d71-4418-baa2-7101c536466b None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:44.133888 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a200d0d-6d71-4418-baa2-7101c536466b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:44.160438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-7a200d0d-6d71-4418-baa2-7101c536466b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:44.161645 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-7a200d0d-6d71-4418-baa2-7101c536466b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:44.163956 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:44.262048 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 31/63] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:44.368327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82612491-11a7-45a2-ba23-a60872b7cc67 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:44.368877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82612491-11a7-45a2-ba23-a60872b7cc67 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:44.369332 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82612491-11a7-45a2-ba23-a60872b7cc67 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:44.370846 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 33/64] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:44.376357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa779281-b25c-404f-9d16-3dc45d88914c None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:44.376647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa779281-b25c-404f-9d16-3dc45d88914c None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:44.376924 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa779281-b25c-404f-9d16-3dc45d88914c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:44.399622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-fa779281-b25c-404f-9d16-3dc45d88914c None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:44.400806 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-fa779281-b25c-404f-9d16-3dc45d88914c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:44.494101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 32/65] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:44.576912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82612491-11a7-45a2-ba23-a60872b7cc67 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:44.603513 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95942de8-e014-4216-a5c6-26a84415f19a None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:44.604412 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95942de8-e014-4216-a5c6-26a84415f19a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:44.604889 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95942de8-e014-4216-a5c6-26a84415f19a None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:44.606531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 33/66] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:44.611801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3ecdc-8e83-4ad8-9519-aebdcdf6a4ec None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:44.612317 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3ecdc-8e83-4ad8-9519-aebdcdf6a4ec None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:44.612782 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3ecdc-8e83-4ad8-9519-aebdcdf6a4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:44.651608 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-42b3ecdc-8e83-4ad8-9519-aebdcdf6a4ec None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:44.652830 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-42b3ecdc-8e83-4ad8-9519-aebdcdf6a4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:44.734345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82612491-11a7-45a2-ba23-a60872b7cc67 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:44.737986 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:44.745112 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 34/67] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:44.746227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffe699-2ec1-4603-86e7-2f19e9eec058 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:44.746483 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffe699-2ec1-4603-86e7-2f19e9eec058 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:44.746985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffe699-2ec1-4603-86e7-2f19e9eec058 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:44.748247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 35/68] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:44.749447 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff5b1b-f031-4bcc-9644-21beb3678af9 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:44.749922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff5b1b-f031-4bcc-9644-21beb3678af9 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:44.750386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff5b1b-f031-4bcc-9644-21beb3678af9 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:44.752050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 36/69] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:44.753491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53761f2-00ec-4229-b461-7aeb86ff9ed1 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:44.753847 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53761f2-00ec-4229-b461-7aeb86ff9ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:44.754209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53761f2-00ec-4229-b461-7aeb86ff9ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:44.773583 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-e53761f2-00ec-4229-b461-7aeb86ff9ed1 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:44.775750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e53761f2-00ec-4229-b461-7aeb86ff9ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:44.785584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff4394-c7a5-4854-a7b4-3631652c9030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:44.786229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff4394-c7a5-4854-a7b4-3631652c9030 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:44.786509 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff4394-c7a5-4854-a7b4-3631652c9030 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:44.786783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff4394-c7a5-4854-a7b4-3631652c9030 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:44.799872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff4394-c7a5-4854-a7b4-3631652c9030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:44.806616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-3aff4394-c7a5-4854-a7b4-3631652c9030 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:58:44.806616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:44.806616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:44.806616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:44.806616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:44.806616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:44.806616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:44.806616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:44.806616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:44.806616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:44.806616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:44.806616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:44.806616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:44.806616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:44.806616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:44.806616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:44.807718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:44.807718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:44.807718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:44.807718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:44.807718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:44.807718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:44.807718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:44.807718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:44.807718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:44.807718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:44.807718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:44.807718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:44.807718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:44.807718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:44.807718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:44.807718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:44.808507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:44.808507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:44.808507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:44.808507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:44.808507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:44.808507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:44.808507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:44.808507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:44.808507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:44.808507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:44.808507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:58:44.808507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:58:44.811779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 34/70] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:58:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:58:44.813543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c81fb-0617-4f06-aacc-cd07a644422b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:44.814045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c81fb-0617-4f06-aacc-cd07a644422b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:44.814327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c81fb-0617-4f06-aacc-cd07a644422b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:44.835662 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-532c81fb-0617-4f06-aacc-cd07a644422b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:44.836985 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-532c81fb-0617-4f06-aacc-cd07a644422b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:44.862098 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:44.870323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 37/71] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:58:44.871002 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53761f2-00ec-4229-b461-7aeb86ff9ed1 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:44.931981 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 35/72] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:44.931981 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c81fb-0617-4f06-aacc-cd07a644422b None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:45.012247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53761f2-00ec-4229-b461-7aeb86ff9ed1 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:45.048603 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:45.059367 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0c111-3bbc-49fc-8dd3-8583921f2910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:45.060336 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0c111-3bbc-49fc-8dd3-8583921f2910 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:45.060683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0c111-3bbc-49fc-8dd3-8583921f2910 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:45.061301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0c111-3bbc-49fc-8dd3-8583921f2910 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:45.062798 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0c111-3bbc-49fc-8dd3-8583921f2910 None admin] RBAC: Adding query filter params (name=member) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:45.063171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0c111-3bbc-49fc-8dd3-8583921f2910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:45.064707 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0c111-3bbc-49fc-8dd3-8583921f2910 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:45.074931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 38/73] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:58:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:45.075843 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0c111-3bbc-49fc-8dd3-8583921f2910 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:45.080905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c81fb-0617-4f06-aacc-cd07a644422b None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:45.120813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9cbfc-0d2e-4096-847f-ffe6d5c772d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:45.121962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9cbfc-0d2e-4096-847f-ffe6d5c772d3 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:45.122250 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9cbfc-0d2e-4096-847f-ffe6d5c772d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:45.122838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9cbfc-0d2e-4096-847f-ffe6d5c772d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:45.138948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9cbfc-0d2e-4096-847f-ffe6d5c772d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:45.143738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-dbd9cbfc-0d2e-4096-847f-ffe6d5c772d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:58:45.143738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:45.143738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:45.143738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:45.143738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:45.143738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:45.143738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:45.143738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:45.143738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:45.143738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:45.143738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:45.143738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:45.143738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:45.143738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:45.143738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:45.143738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:45.144745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:45.144745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:45.144745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:45.144745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:45.144745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:45.144745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:45.144745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:45.144745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:45.144745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:45.144745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:45.144745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:45.144745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:45.144745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:45.144745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:45.144745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:45.144745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:45.145511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:45.145511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:45.145511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:45.145511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:45.145511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:45.145511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:45.145511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:45.145511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:45.145511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:45.145511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:45.145511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:58:45.145511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:58:45.145511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 36/74] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 06:58:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:58:45.146137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9cbfc-0d2e-4096-847f-ffe6d5c772d3 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:45.171515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:45.183726 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0c111-3bbc-49fc-8dd3-8583921f2910 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:45.220022 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae4c73-c4e9-4048-ab26-8e5d6f6be143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:45.221012 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae4c73-c4e9-4048-ab26-8e5d6f6be143 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:45.221012 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae4c73-c4e9-4048-ab26-8e5d6f6be143 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:45.221203 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae4c73-c4e9-4048-ab26-8e5d6f6be143 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:45.258272 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9cbfc-0d2e-4096-847f-ffe6d5c772d3 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:45.295299 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a2570-1a01-4131-bf9e-46489f665b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:45.295984 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a2570-1a01-4131-bf9e-46489f665b01 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:45.296267 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a2570-1a01-4131-bf9e-46489f665b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:45.296545 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a2570-1a01-4131-bf9e-46489f665b01 None admin] PATH_INFO: `/v3/users/1eb74ccd914b40dbac0acc5a1d975c76` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:45.312796 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a2570-1a01-4131-bf9e-46489f665b01 None admin] RBAC: Authorizing `identity:get_user(user_id=1eb74ccd914b40dbac0acc5a1d975c76)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:45.314328 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a2570-1a01-4131-bf9e-46489f665b01 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:45.316108 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 37/75] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:58:45 2022] GET /identity/v3/users/1eb74ccd914b40dbac0acc5a1d975c76 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:45.316922 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a2570-1a01-4131-bf9e-46489f665b01 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:45.340353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae4c73-c4e9-4048-ab26-8e5d6f6be143 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:45.346243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-84ae4c73-c4e9-4048-ab26-8e5d6f6be143 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:58:45.346243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:45.346243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:45.346243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:45.346243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:45.346243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:45.346243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:45.346243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:45.346243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:45.346243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:45.346243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:45.346243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:45.346243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:45.346243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:45.346243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:45.346243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:45.347130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:45.347130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:45.347130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:45.347130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:45.347130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:45.347130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:45.347130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:45.347130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:45.347130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:45.347130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:45.347130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:45.347130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:45.347130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:45.347130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:45.347130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:45.347130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:45.347922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:45.347922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:45.347922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:45.347922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:45.347922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:45.347922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:45.347922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:45.347922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:45.347922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:45.347922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:45.347922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:58:45.347922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:58:45.347922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 39/76] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:58:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:58:45.348555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae4c73-c4e9-4048-ab26-8e5d6f6be143 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:45.403727 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:45.408243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:45.426626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a2570-1a01-4131-bf9e-46489f665b01 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:45.458814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae4c73-c4e9-4048-ab26-8e5d6f6be143 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:45.461562 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8393d-5ab3-4a25-a7f6-e1fa932e2970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:45.462192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8393d-5ab3-4a25-a7f6-e1fa932e2970 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:45.462486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8393d-5ab3-4a25-a7f6-e1fa932e2970 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:45.462767 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8393d-5ab3-4a25-a7f6-e1fa932e2970 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:45.472765 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8393d-5ab3-4a25-a7f6-e1fa932e2970 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:45.477565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-02e8393d-5ab3-4a25-a7f6-e1fa932e2970 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:58:45.477565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:45.477565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:45.477565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:45.477565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:45.477565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:45.477565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:45.477565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:45.477565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:45.477565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:45.477565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:45.477565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:45.477565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:45.477565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:45.477565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:45.477565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:45.478435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:45.478435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:45.478435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:45.478435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:45.478435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:45.478435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:45.478435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:45.478435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:45.478435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:45.478435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:45.478435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:45.478435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:45.478435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:45.478435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:45.478435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:45.478435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:45.479193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:45.479193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:45.479193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:45.479193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:45.479193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:45.479193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:45.479193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:45.479193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:45.479193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:45.479193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:45.479193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:58:45.479193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:58:45.479193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 38/77] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 06:58:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:58:45.479917 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8393d-5ab3-4a25-a7f6-e1fa932e2970 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:45.498464 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf43bc5-1ecd-431d-a58f-0527dd645f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:45.499086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf43bc5-1ecd-431d-a58f-0527dd645f16 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:45.499369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf43bc5-1ecd-431d-a58f-0527dd645f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:45.499751 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf43bc5-1ecd-431d-a58f-0527dd645f16 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:45.500445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf43bc5-1ecd-431d-a58f-0527dd645f16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:45.500752 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf43bc5-1ecd-431d-a58f-0527dd645f16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:45.502173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf43bc5-1ecd-431d-a58f-0527dd645f16 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:45.511669 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 40/78] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:58:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:45.512353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf43bc5-1ecd-431d-a58f-0527dd645f16 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:45.578191 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8393d-5ab3-4a25-a7f6-e1fa932e2970 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:45.611258 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915369a6-9dfd-4e21-822c-0b9e41c5b5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:45.611885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915369a6-9dfd-4e21-822c-0b9e41c5b5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:45.612169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915369a6-9dfd-4e21-822c-0b9e41c5b5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:45.612478 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915369a6-9dfd-4e21-822c-0b9e41c5b5c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:45.613616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915369a6-9dfd-4e21-822c-0b9e41c5b5c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:45.615091 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915369a6-9dfd-4e21-822c-0b9e41c5b5c6 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:45.615510 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf43bc5-1ecd-431d-a58f-0527dd645f16 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:45.616869 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 39/79] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:58:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:45.617641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915369a6-9dfd-4e21-822c-0b9e41c5b5c6 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:45.627896 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:45.629542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:45.652419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530a005-3c5f-4d00-b766-6f9319594c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:45.653037 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530a005-3c5f-4d00-b766-6f9319594c37 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:45.653319 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530a005-3c5f-4d00-b766-6f9319594c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:45.653592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530a005-3c5f-4d00-b766-6f9319594c37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:45.722895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530a005-3c5f-4d00-b766-6f9319594c37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:45.724422 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530a005-3c5f-4d00-b766-6f9319594c37 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:45.724526 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915369a6-9dfd-4e21-822c-0b9e41c5b5c6 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:45.758538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee0fc7-e27b-473a-b1ba-ea06f46fe371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:45.759153 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee0fc7-e27b-473a-b1ba-ea06f46fe371 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:45.759442 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee0fc7-e27b-473a-b1ba-ea06f46fe371 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:45.759741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee0fc7-e27b-473a-b1ba-ea06f46fe371 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:45.760418 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee0fc7-e27b-473a-b1ba-ea06f46fe371 None admin] RBAC: Adding query filter params (name=member) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:45.760726 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee0fc7-e27b-473a-b1ba-ea06f46fe371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:45.762148 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee0fc7-e27b-473a-b1ba-ea06f46fe371 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:45.770361 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 40/80] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:58:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:45.771148 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee0fc7-e27b-473a-b1ba-ea06f46fe371 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:45.828251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 41/81] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:58:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:58:45.829096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530a005-3c5f-4d00-b766-6f9319594c37 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:45.836815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:45.843619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:45.871730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee0fc7-e27b-473a-b1ba-ea06f46fe371 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:45.909776 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e819005-a94e-4f9e-8008-b9986418857f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:45.910395 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e819005-a94e-4f9e-8008-b9986418857f None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:45.910676 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e819005-a94e-4f9e-8008-b9986418857f None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:45.910945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e819005-a94e-4f9e-8008-b9986418857f None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:45.911649 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e819005-a94e-4f9e-8008-b9986418857f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:45.911952 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e819005-a94e-4f9e-8008-b9986418857f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:45.913495 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e819005-a94e-4f9e-8008-b9986418857f None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:45.921871 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 41/82] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:58:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:45.922633 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e819005-a94e-4f9e-8008-b9986418857f None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:45.930205 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530a005-3c5f-4d00-b766-6f9319594c37 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:45.972207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1dff3-e13f-46e9-b61d-bcea73510dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:45.972838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1dff3-e13f-46e9-b61d-bcea73510dad None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:45.973182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1dff3-e13f-46e9-b61d-bcea73510dad None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:45.973395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1dff3-e13f-46e9-b61d-bcea73510dad None admin] PATH_INFO: `/v3/users/2ff4f07bfac14f428ece0dfbc5ec3579` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:45.974520 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1dff3-e13f-46e9-b61d-bcea73510dad None admin] RBAC: Authorizing `identity:get_user(user_id=2ff4f07bfac14f428ece0dfbc5ec3579)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:45.976150 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1dff3-e13f-46e9-b61d-bcea73510dad None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:45.977733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 42/83] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:58:45 2022] GET /identity/v3/users/2ff4f07bfac14f428ece0dfbc5ec3579 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:45.978478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1dff3-e13f-46e9-b61d-bcea73510dad None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:46.029230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e819005-a94e-4f9e-8008-b9986418857f None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:46.057344 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:46.060728 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:46.064376 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be468c33-65fd-4c29-a106-bba00712e420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:46.065000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be468c33-65fd-4c29-a106-bba00712e420 None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:46.065283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be468c33-65fd-4c29-a106-bba00712e420 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:46.065557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be468c33-65fd-4c29-a106-bba00712e420 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:46.066209 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be468c33-65fd-4c29-a106-bba00712e420 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:46.067724 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be468c33-65fd-4c29-a106-bba00712e420 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:46.080691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1dff3-e13f-46e9-b61d-bcea73510dad None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:46.097034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 42/84] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 23 06:58:45 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:58:46.097868 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be468c33-65fd-4c29-a106-bba00712e420 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:46.118110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbaf60-a6b0-427d-86f2-d3ab82800a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:46.118765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbaf60-a6b0-427d-86f2-d3ab82800a21 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:46.119041 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbaf60-a6b0-427d-86f2-d3ab82800a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:46.119314 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbaf60-a6b0-427d-86f2-d3ab82800a21 None admin] PATH_INFO: `/v3/users/1eb74ccd914b40dbac0acc5a1d975c76` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:46.120557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcbaf60-a6b0-427d-86f2-d3ab82800a21 None admin] RBAC: Authorizing `identity:get_user(user_id=1eb74ccd914b40dbac0acc5a1d975c76)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:46.122109 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcbaf60-a6b0-427d-86f2-d3ab82800a21 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:46.124178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 43/85] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:58:45 2022] GET /identity/v3/users/1eb74ccd914b40dbac0acc5a1d975c76 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:46.125012 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbaf60-a6b0-427d-86f2-d3ab82800a21 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:46.235744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbaf60-a6b0-427d-86f2-d3ab82800a21 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:46.239463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be468c33-65fd-4c29-a106-bba00712e420 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:46.275152 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf5f31-016e-4152-902b-7a31fa478194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:46.275794 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf5f31-016e-4152-902b-7a31fa478194 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:46.276079 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf5f31-016e-4152-902b-7a31fa478194 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:46.276353 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf5f31-016e-4152-902b-7a31fa478194 None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:46.277488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf5f31-016e-4152-902b-7a31fa478194 None admin] RBAC: Authorizing `identity:get_project(project_id=92ca046a717e4adab11f7371ad3cc888)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:46.278967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf5f31-016e-4152-902b-7a31fa478194 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:46.280351 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32980620-433b-4228-b82d-7e73b19ff4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:46.280899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 43/86] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:58:46 2022] GET /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:46.281009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32980620-433b-4228-b82d-7e73b19ff4ee None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:46.281261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32980620-433b-4228-b82d-7e73b19ff4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:46.281440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf5f31-016e-4152-902b-7a31fa478194 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:46.281535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32980620-433b-4228-b82d-7e73b19ff4ee None admin] PATH_INFO: `/v3/users/1eb74ccd914b40dbac0acc5a1d975c76` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:46.282700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32980620-433b-4228-b82d-7e73b19ff4ee None admin] RBAC: Authorizing `identity:get_user(user_id=1eb74ccd914b40dbac0acc5a1d975c76)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:46.284203 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32980620-433b-4228-b82d-7e73b19ff4ee None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:46.285799 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 44/87] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:58:46 2022] GET /identity/v3/users/1eb74ccd914b40dbac0acc5a1d975c76 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:46.286564 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32980620-433b-4228-b82d-7e73b19ff4ee None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:46.306161 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:46.354824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:46.429766 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32980620-433b-4228-b82d-7e73b19ff4ee None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:46.433938 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf5f31-016e-4152-902b-7a31fa478194 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:46.485634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e9080-60ba-41be-88aa-7cf86efa94b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:46.486698 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e9080-60ba-41be-88aa-7cf86efa94b2 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:46.487110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e9080-60ba-41be-88aa-7cf86efa94b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:46.487607 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e9080-60ba-41be-88aa-7cf86efa94b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:46.499438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35247ddc-622c-4eed-b8ca-3b928e50677f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:46.500402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35247ddc-622c-4eed-b8ca-3b928e50677f None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:46.500854 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35247ddc-622c-4eed-b8ca-3b928e50677f None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:46.501304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35247ddc-622c-4eed-b8ca-3b928e50677f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:46.605699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e9080-60ba-41be-88aa-7cf86efa94b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:46.609294 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e9080-60ba-41be-88aa-7cf86efa94b2 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:46.617764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35247ddc-622c-4eed-b8ca-3b928e50677f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:46.620314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35247ddc-622c-4eed-b8ca-3b928e50677f None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:46.663918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:46.696497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:46.748627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 44/88] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:58:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:58:46.749528 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35247ddc-622c-4eed-b8ca-3b928e50677f None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:46.770169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 45/89] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:58:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:58:46.771258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e9080-60ba-41be-88aa-7cf86efa94b2 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:46.860630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35247ddc-622c-4eed-b8ca-3b928e50677f None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:46.893259 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e9080-60ba-41be-88aa-7cf86efa94b2 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:46.907569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9450cb-4059-4c04-8888-1268ca6763df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:46.908433 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9450cb-4059-4c04-8888-1268ca6763df None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:46.909167 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9450cb-4059-4c04-8888-1268ca6763df None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:46.909449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9450cb-4059-4c04-8888-1268ca6763df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:46.911652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:46.933058 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416b178-16e4-4337-b27c-9b0c8559045e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:46.933789 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416b178-16e4-4337-b27c-9b0c8559045e None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:46.934078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416b178-16e4-4337-b27c-9b0c8559045e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:46.934461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416b178-16e4-4337-b27c-9b0c8559045e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:46.935205 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2416b178-16e4-4337-b27c-9b0c8559045e None admin] RBAC: Adding query filter params (role.id=7670c7c450b9404ba57e12743cb6c458, scope.project.id=92ca046a717e4adab11f7371ad3cc888, user.id=1eb74ccd914b40dbac0acc5a1d975c76) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:46.935560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2416b178-16e4-4337-b27c-9b0c8559045e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:46.937060 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2416b178-16e4-4337-b27c-9b0c8559045e None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:46.940661 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:46.948800 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 46/90] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:58:46 2022] GET /identity/v3/role_assignments?user.id=1eb74ccd914b40dbac0acc5a1d975c76&scope.project.id=92ca046a717e4adab11f7371ad3cc888&role.id=7670c7c450b9404ba57e12743cb6c458 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:46.949701 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416b178-16e4-4337-b27c-9b0c8559045e None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:47.036307 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9450cb-4059-4c04-8888-1268ca6763df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:47.037833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9450cb-4059-4c04-8888-1268ca6763df None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:47.170500 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416b178-16e4-4337-b27c-9b0c8559045e None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:47.222534 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 45/91] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:58:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:58:47.223235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9450cb-4059-4c04-8888-1268ca6763df None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:47.231760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808a30f-4b17-4c5a-b657-c2ca7253ed6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:47.232721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808a30f-4b17-4c5a-b657-c2ca7253ed6b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:47.233234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808a30f-4b17-4c5a-b657-c2ca7253ed6b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:47.233739 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808a30f-4b17-4c5a-b657-c2ca7253ed6b None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:47.235071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808a30f-4b17-4c5a-b657-c2ca7253ed6b None admin] RBAC: Authorizing `identity:get_project(project_id=92ca046a717e4adab11f7371ad3cc888)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:47.243436 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808a30f-4b17-4c5a-b657-c2ca7253ed6b None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:47.245860 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 47/92] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:58:46 2022] GET /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:47.246813 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808a30f-4b17-4c5a-b657-c2ca7253ed6b None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:47.291541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:47.361354 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9450cb-4059-4c04-8888-1268ca6763df None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:47.371883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:47.451398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808a30f-4b17-4c5a-b657-c2ca7253ed6b None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:47.477052 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8043513-6614-4476-a4a1-fb552b710cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:47.477723 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8043513-6614-4476-a4a1-fb552b710cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:47.478008 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8043513-6614-4476-a4a1-fb552b710cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:47.478292 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8043513-6614-4476-a4a1-fb552b710cf7 None admin] PATH_INFO: `/v3/projects/96f9988774414935ad8f1a88bef57274` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:47.479457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8043513-6614-4476-a4a1-fb552b710cf7 None admin] RBAC: Authorizing `identity:get_project(project_id=96f9988774414935ad8f1a88bef57274)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:47.480948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8043513-6614-4476-a4a1-fb552b710cf7 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:47.482759 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 46/93] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:58:47 2022] GET /identity/v3/projects/96f9988774414935ad8f1a88bef57274 => generated 312 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:47.485134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8043513-6614-4476-a4a1-fb552b710cf7 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:47.492856 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed4a4b-21c4-4dc7-9448-86da0482a680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:47.494358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed4a4b-21c4-4dc7-9448-86da0482a680 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:47.494652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed4a4b-21c4-4dc7-9448-86da0482a680 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:47.494923 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed4a4b-21c4-4dc7-9448-86da0482a680 None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:47.499274 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed4a4b-21c4-4dc7-9448-86da0482a680 None admin] RBAC: Authorizing `identity:get_project(project_id=92ca046a717e4adab11f7371ad3cc888)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:47.501178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed4a4b-21c4-4dc7-9448-86da0482a680 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:47.503508 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 48/94] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:58:47 2022] GET /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:47.504293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed4a4b-21c4-4dc7-9448-86da0482a680 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:47.668630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8043513-6614-4476-a4a1-fb552b710cf7 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:47.703253 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed4a4b-21c4-4dc7-9448-86da0482a680 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:47.718787 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:47.736264 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:47.737216 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252aa78-2f6c-416d-a58d-5f654a233eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:47.737835 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252aa78-2f6c-416d-a58d-5f654a233eba None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:47.738218 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252aa78-2f6c-416d-a58d-5f654a233eba None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:47.742724 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252aa78-2f6c-416d-a58d-5f654a233eba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:47.743542 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252aa78-2f6c-416d-a58d-5f654a233eba None admin] RBAC: Adding query filter params (role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.project.id=92ca046a717e4adab11f7371ad3cc888, user.id=2ff4f07bfac14f428ece0dfbc5ec3579) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:47.743860 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252aa78-2f6c-416d-a58d-5f654a233eba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:47.745358 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252aa78-2f6c-416d-a58d-5f654a233eba None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:47.758488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 47/95] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:58:47 2022] GET /identity/v3/role_assignments?user.id=2ff4f07bfac14f428ece0dfbc5ec3579&scope.project.id=92ca046a717e4adab11f7371ad3cc888&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:47.759363 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252aa78-2f6c-416d-a58d-5f654a233eba None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:47.775913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b07181-c6bb-45a9-bcb6-350344e7945b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:47.776538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b07181-c6bb-45a9-bcb6-350344e7945b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:47.776815 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b07181-c6bb-45a9-bcb6-350344e7945b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:47.777089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b07181-c6bb-45a9-bcb6-350344e7945b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:47.777826 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b07181-c6bb-45a9-bcb6-350344e7945b None admin] RBAC: Adding query filter params (role.id=7670c7c450b9404ba57e12743cb6c458, scope.project.id=96f9988774414935ad8f1a88bef57274, user.id=1eb74ccd914b40dbac0acc5a1d975c76) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:47.778131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b07181-c6bb-45a9-bcb6-350344e7945b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:47.779609 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b07181-c6bb-45a9-bcb6-350344e7945b None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:47.795463 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 49/96] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:58:47 2022] GET /identity/v3/role_assignments?user.id=1eb74ccd914b40dbac0acc5a1d975c76&scope.project.id=96f9988774414935ad8f1a88bef57274&role.id=7670c7c450b9404ba57e12743cb6c458 => generated 259 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:47.915571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252aa78-2f6c-416d-a58d-5f654a233eba None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:47.969465 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96203f-2e8e-4663-83f5-d190e00a0c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:47.970104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96203f-2e8e-4663-83f5-d190e00a0c0b None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:47.970386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96203f-2e8e-4663-83f5-d190e00a0c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:47.970665 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96203f-2e8e-4663-83f5-d190e00a0c0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:47.971445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96203f-2e8e-4663-83f5-d190e00a0c0b None admin] RBAC: Adding query filter params (role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7, scope.project.id=92ca046a717e4adab11f7371ad3cc888, user.id=1eb74ccd914b40dbac0acc5a1d975c76) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:47.971777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96203f-2e8e-4663-83f5-d190e00a0c0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:47.973321 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96203f-2e8e-4663-83f5-d190e00a0c0b None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:47.984555 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 48/97] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:58:47 2022] GET /identity/v3/role_assignments?user.id=1eb74ccd914b40dbac0acc5a1d975c76&scope.project.id=92ca046a717e4adab11f7371ad3cc888&role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:48.366112 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3229a1b-0288-4e94-9db3-d469a4e64a1b None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:48.366691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3229a1b-0288-4e94-9db3-d469a4e64a1b None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:48.366691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3229a1b-0288-4e94-9db3-d469a4e64a1b None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:48.368820 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 50/98] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:48.374882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a054817-0bc4-498c-b46d-fb72bd6e6589 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:48.375839 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a054817-0bc4-498c-b46d-fb72bd6e6589 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:48.376322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a054817-0bc4-498c-b46d-fb72bd6e6589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:48.404596 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-2a054817-0bc4-498c-b46d-fb72bd6e6589 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:48.406261 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2a054817-0bc4-498c-b46d-fb72bd6e6589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:48.469892 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:48.498661 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 49/99] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:48.635554 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388874c3-9d17-4e15-9dd6-e399c3ca9c50 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:48.636186 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388874c3-9d17-4e15-9dd6-e399c3ca9c50 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:48.636665 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388874c3-9d17-4e15-9dd6-e399c3ca9c50 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:48.639321 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 51/100] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:48.647100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408de51-3f31-4c6f-bd26-691156a6cdae None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:48.647818 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408de51-3f31-4c6f-bd26-691156a6cdae None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:48.648449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408de51-3f31-4c6f-bd26-691156a6cdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:48.674488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-f408de51-3f31-4c6f-bd26-691156a6cdae None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:48.675734 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f408de51-3f31-4c6f-bd26-691156a6cdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:48.772865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 50/101] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:49.009694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388874c3-9d17-4e15-9dd6-e399c3ca9c50 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:49.142143 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388874c3-9d17-4e15-9dd6-e399c3ca9c50 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:49.175823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d6b51-579d-4e90-9f21-8ea82b398f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:49.176466 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d6b51-579d-4e90-9f21-8ea82b398f88 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:49.176813 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d6b51-579d-4e90-9f21-8ea82b398f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:49.177093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d6b51-579d-4e90-9f21-8ea82b398f88 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:49.189058 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d6b51-579d-4e90-9f21-8ea82b398f88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:49.194010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-678d6b51-579d-4e90-9f21-8ea82b398f88 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 06:58:49.194010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:49.194010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:49.194010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:49.194010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:49.194010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:49.194010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:49.194010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:49.194010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:49.194010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:49.194010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:49.194010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:49.194010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:49.194010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:49.194010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:49.194010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:49.194905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:49.194905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:49.194905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:49.194905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:49.194905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:49.194905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:49.194905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:49.194905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:49.194905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:49.194905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:49.194905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:49.194905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:49.194905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:49.194905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:49.194905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:49.194905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:49.195772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:49.195772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:49.195772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:49.195772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:49.195772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:49.195772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:49.195772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:49.195772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:49.195772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:49.195772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:49.195772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 06:58:49.195772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:58:49.195772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 52/102] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:58:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:58:49.201431 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408de51-3f31-4c6f-bd26-691156a6cdae None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:49.333906 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408de51-3f31-4c6f-bd26-691156a6cdae None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:49.346396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:49.378937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abba79-ee9d-4a73-a521-750bde675bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:49.379579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abba79-ee9d-4a73-a521-750bde675bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:49.379871 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abba79-ee9d-4a73-a521-750bde675bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:49.380146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abba79-ee9d-4a73-a521-750bde675bd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:49.380836 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1abba79-ee9d-4a73-a521-750bde675bd2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:49.381139 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1abba79-ee9d-4a73-a521-750bde675bd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:49.382738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1abba79-ee9d-4a73-a521-750bde675bd2 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:49.395136 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 51/103] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:58:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:49.404543 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d6b51-579d-4e90-9f21-8ea82b398f88 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:49.412646 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abba79-ee9d-4a73-a521-750bde675bd2 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:49.495670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:49.610170 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d6b51-579d-4e90-9f21-8ea82b398f88 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:49.641696 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abba79-ee9d-4a73-a521-750bde675bd2 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:49.644517 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d33bb-8610-4712-94f9-e32d74a00f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:49.645185 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d33bb-8610-4712-94f9-e32d74a00f91 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:49.645462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d33bb-8610-4712-94f9-e32d74a00f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:49.645736 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d33bb-8610-4712-94f9-e32d74a00f91 None admin] PATH_INFO: `/v3/users/dbc5ca4b49e54cb7ab4206845a8b8e18` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:49.661344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d33bb-8610-4712-94f9-e32d74a00f91 None admin] RBAC: Authorizing `identity:get_user(user_id=dbc5ca4b49e54cb7ab4206845a8b8e18)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:49.662941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d33bb-8610-4712-94f9-e32d74a00f91 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:49.664754 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 53/104] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:58:49 2022] GET /identity/v3/users/dbc5ca4b49e54cb7ab4206845a8b8e18 => generated 289 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:49.665919 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a8a70-450e-4cc8-acf0-819f16b49496 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:49.666200 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a8a70-450e-4cc8-acf0-819f16b49496 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:49.666476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a8a70-450e-4cc8-acf0-819f16b49496 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:49.672234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 54/105] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:49.672972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a8a70-450e-4cc8-acf0-819f16b49496 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:49.760131 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230c417-f959-4fb7-aabe-30f0ec3486e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:49.760788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230c417-f959-4fb7-aabe-30f0ec3486e7 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:49.761074 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230c417-f959-4fb7-aabe-30f0ec3486e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:49.761351 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230c417-f959-4fb7-aabe-30f0ec3486e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:49.768120 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230c417-f959-4fb7-aabe-30f0ec3486e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:49.769619 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230c417-f959-4fb7-aabe-30f0ec3486e7 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:49.771540 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 52/106] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:58:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:58:49.772685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00692d8f-f462-4f2f-96fc-68175da4b52c None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:49.772999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00692d8f-f462-4f2f-96fc-68175da4b52c None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:49.773930 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00692d8f-f462-4f2f-96fc-68175da4b52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:49.774398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:49.784762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a8a70-450e-4cc8-acf0-819f16b49496 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:49.810644 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-00692d8f-f462-4f2f-96fc-68175da4b52c None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:49.811868 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-00692d8f-f462-4f2f-96fc-68175da4b52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:49.852142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333133c9-8359-4f71-9e40-1f1ae547873d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:49.852805 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333133c9-8359-4f71-9e40-1f1ae547873d None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:49.853043 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333133c9-8359-4f71-9e40-1f1ae547873d None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:49.853268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333133c9-8359-4f71-9e40-1f1ae547873d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:49.877915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:49.915160 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 53/107] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:49.916578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00692d8f-f462-4f2f-96fc-68175da4b52c None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:50.038918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333133c9-8359-4f71-9e40-1f1ae547873d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:50.040853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333133c9-8359-4f71-9e40-1f1ae547873d None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:50.056030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00692d8f-f462-4f2f-96fc-68175da4b52c None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:50.093106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8067ed6-774d-44a5-98f9-c4c8aaaf5b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:50.093733 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8067ed6-774d-44a5-98f9-c4c8aaaf5b20 None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:50.094013 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8067ed6-774d-44a5-98f9-c4c8aaaf5b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:50.094290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8067ed6-774d-44a5-98f9-c4c8aaaf5b20 None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:50.099372 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8067ed6-774d-44a5-98f9-c4c8aaaf5b20 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:50.099372 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8067ed6-774d-44a5-98f9-c4c8aaaf5b20 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:50.161755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 54/108] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 23 06:58:49 2022] POST /identity/v3/projects => generated 302 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:58:50.208275 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 55/109] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:58:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:58:50.211893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:50.214184 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8067ed6-774d-44a5-98f9-c4c8aaaf5b20 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:50.239309 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333133c9-8359-4f71-9e40-1f1ae547873d None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:50.269367 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:50.433784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8067ed6-774d-44a5-98f9-c4c8aaaf5b20 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:50.463374 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333133c9-8359-4f71-9e40-1f1ae547873d None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:50.498924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4fc02-0bf6-4bcb-a02c-2c82ad6fd288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:50.499607 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4fc02-0bf6-4bcb-a02c-2c82ad6fd288 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:50.499889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4fc02-0bf6-4bcb-a02c-2c82ad6fd288 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:50.500165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4fc02-0bf6-4bcb-a02c-2c82ad6fd288 None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:50.501445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4fc02-0bf6-4bcb-a02c-2c82ad6fd288 None admin] RBAC: Authorizing `identity:get_project(project_id=92ca046a717e4adab11f7371ad3cc888)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:50.502973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4fc02-0bf6-4bcb-a02c-2c82ad6fd288 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:50.505952 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 55/110] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:58:50 2022] GET /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888 => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:50.513629 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4fc02-0bf6-4bcb-a02c-2c82ad6fd288 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:50.528357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52ec4e-9a3f-4bca-bc76-dd3cdfdc6f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:50.529072 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52ec4e-9a3f-4bca-bc76-dd3cdfdc6f51 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:50.529331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52ec4e-9a3f-4bca-bc76-dd3cdfdc6f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:50.529650 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52ec4e-9a3f-4bca-bc76-dd3cdfdc6f51 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:50.551465 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52ec4e-9a3f-4bca-bc76-dd3cdfdc6f51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:50.556722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-be52ec4e-9a3f-4bca-bc76-dd3cdfdc6f51 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:58:50.556722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:50.556722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:50.556722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:50.556722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:50.556722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:50.556722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:50.556722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:50.556722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:50.556722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:50.556722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:50.556722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:50.556722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:50.556722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:50.556722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:50.556722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:50.557592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:50.557592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:50.557592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:50.557592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:50.557592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:50.557592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:50.557592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:50.557592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:50.557592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:50.557592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:50.557592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:50.557592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:50.557592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:50.557592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:50.557592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:50.557592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:50.558350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:50.558350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:50.558350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:50.558350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:50.558350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:50.558350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:50.558350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:50.558350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:50.558350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:50.558350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:50.558350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:58:50.558350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:58:50.559684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 56/111] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:58:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:58:50.562146 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce4975-dc88-450a-a3d4-5740eebb3271 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:50.562429 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce4975-dc88-450a-a3d4-5740eebb3271 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:50.562700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce4975-dc88-450a-a3d4-5740eebb3271 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:50.564314 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 57/112] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:50.565883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce4975-dc88-450a-a3d4-5740eebb3271 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:50.667779 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:50.686057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:50.698354 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4fc02-0bf6-4bcb-a02c-2c82ad6fd288 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:50.717415 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce4975-dc88-450a-a3d4-5740eebb3271 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:50.736892 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf5ce8-d3f4-4c62-8056-31c27d334ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:50.737732 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf5ce8-d3f4-4c62-8056-31c27d334ddd None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:50.737837 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf5ce8-d3f4-4c62-8056-31c27d334ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:50.738060 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf5ce8-d3f4-4c62-8056-31c27d334ddd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:50.738807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf5ce8-d3f4-4c62-8056-31c27d334ddd None admin] RBAC: Adding query filter params (role.id=ff4e7aa3dd7e4272a3cd1cafb32baa24, scope.project.id=92ca046a717e4adab11f7371ad3cc888, user.id=dbc5ca4b49e54cb7ab4206845a8b8e18) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:50.739123 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf5ce8-d3f4-4c62-8056-31c27d334ddd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:50.740627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf5ce8-d3f4-4c62-8056-31c27d334ddd None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:50.746600 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 56/113] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:58:50 2022] GET /identity/v3/role_assignments?user.id=dbc5ca4b49e54cb7ab4206845a8b8e18&scope.project.id=92ca046a717e4adab11f7371ad3cc888&role.id=ff4e7aa3dd7e4272a3cd1cafb32baa24 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:50.747552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57326f-9a5b-4375-a8a8-6047946f34e0 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:50.747903 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57326f-9a5b-4375-a8a8-6047946f34e0 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:50.748118 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57326f-9a5b-4375-a8a8-6047946f34e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:50.760499 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ecb89-22ba-4421-a771-98889e82aac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:50.761499 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ecb89-22ba-4421-a771-98889e82aac6 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:50.761994 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ecb89-22ba-4421-a771-98889e82aac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:50.763228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ecb89-22ba-4421-a771-98889e82aac6 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:50.764251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ecb89-22ba-4421-a771-98889e82aac6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:50.765027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ecb89-22ba-4421-a771-98889e82aac6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:50.766716 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ecb89-22ba-4421-a771-98889e82aac6 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:50.768902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-4c57326f-9a5b-4375-a8a8-6047946f34e0 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:50.770373 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-4c57326f-9a5b-4375-a8a8-6047946f34e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:50.779318 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 58/114] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:58:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:50.780931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc93a0-d65c-478d-b78d-5cca2813bbc9 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:50.781356 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc93a0-d65c-478d-b78d-5cca2813bbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:50.781788 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc93a0-d65c-478d-b78d-5cca2813bbc9 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:50.783290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 59/115] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:50.785633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64728071-b458-4b6d-a87f-155e6acf94d5 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:50.786060 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64728071-b458-4b6d-a87f-155e6acf94d5 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:50.786468 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64728071-b458-4b6d-a87f-155e6acf94d5 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:50.788306 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 60/116] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:50.789391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64728071-b458-4b6d-a87f-155e6acf94d5 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:50.868103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 57/117] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:50.869499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2381934-7ec5-4d2e-a56c-aadddfb81dcd None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:50.870043 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2381934-7ec5-4d2e-a56c-aadddfb81dcd None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:50.870155 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2381934-7ec5-4d2e-a56c-aadddfb81dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:50.890379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-c2381934-7ec5-4d2e-a56c-aadddfb81dcd None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:50.891676 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c2381934-7ec5-4d2e-a56c-aadddfb81dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:50.932031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64728071-b458-4b6d-a87f-155e6acf94d5 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:50.961897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:50.986140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 58/118] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:50.987440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462b61e-7e7c-4294-800b-e0e292a1d9c3 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:50.987733 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462b61e-7e7c-4294-800b-e0e292a1d9c3 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:50.988146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462b61e-7e7c-4294-800b-e0e292a1d9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:51.000152 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac06b11-5ddf-4e83-a101-0d309184c7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:51.001101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac06b11-5ddf-4e83-a101-0d309184c7ec None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:51.001101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac06b11-5ddf-4e83-a101-0d309184c7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:51.001362 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac06b11-5ddf-4e83-a101-0d309184c7ec None admin] PATH_INFO: `/v3/users/1eb74ccd914b40dbac0acc5a1d975c76` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:51.002529 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac06b11-5ddf-4e83-a101-0d309184c7ec None admin] RBAC: Authorizing `identity:get_user(user_id=1eb74ccd914b40dbac0acc5a1d975c76)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:51.004113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac06b11-5ddf-4e83-a101-0d309184c7ec None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:51.005876 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 61/119] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:58:50 2022] GET /identity/v3/users/1eb74ccd914b40dbac0acc5a1d975c76 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:51.020428 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac06b11-5ddf-4e83-a101-0d309184c7ec None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:51.024610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-8462b61e-7e7c-4294-800b-e0e292a1d9c3 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:51.025926 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8462b61e-7e7c-4294-800b-e0e292a1d9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:51.044204 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:51.177056 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 59/120] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:51.177652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac06b11-5ddf-4e83-a101-0d309184c7ec None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:51.177992 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462b61e-7e7c-4294-800b-e0e292a1d9c3 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:51.214824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85703b-4a0b-42b4-b669-f01c7fa6b6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:51.215387 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85703b-4a0b-42b4-b669-f01c7fa6b6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:51.215711 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85703b-4a0b-42b4-b669-f01c7fa6b6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:51.216030 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85703b-4a0b-42b4-b669-f01c7fa6b6e7 None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:51.217187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85703b-4a0b-42b4-b669-f01c7fa6b6e7 None admin] RBAC: Authorizing `identity:get_project(project_id=92ca046a717e4adab11f7371ad3cc888)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:51.218725 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85703b-4a0b-42b4-b669-f01c7fa6b6e7 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:51.220584 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 62/121] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:58:51 2022] GET /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:58:51.226149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85703b-4a0b-42b4-b669-f01c7fa6b6e7 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:51.285260 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:51.329866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85703b-4a0b-42b4-b669-f01c7fa6b6e7 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:51.369310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462b61e-7e7c-4294-800b-e0e292a1d9c3 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:51.389330 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c00ee2-1f6a-4d73-bd84-086648822ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:51.390017 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c00ee2-1f6a-4d73-bd84-086648822ff9 None admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:51.390302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c00ee2-1f6a-4d73-bd84-086648822ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:51.390580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c00ee2-1f6a-4d73-bd84-086648822ff9 None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888/users/1eb74ccd914b40dbac0acc5a1d975c76/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:51.392619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c00ee2-1f6a-4d73-bd84-086648822ff9 None admin] RBAC: Authorizing `identity:create_grant(project_id=92ca046a717e4adab11f7371ad3cc888, user_id=1eb74ccd914b40dbac0acc5a1d975c76, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:51.394105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c00ee2-1f6a-4d73-bd84-086648822ff9 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:51.407562 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac9a21-9cdc-422d-bdef-e4f22b8ed4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:51.408541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac9a21-9cdc-422d-bdef-e4f22b8ed4ef None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:51.409032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac9a21-9cdc-422d-bdef-e4f22b8ed4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:51.409491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac9a21-9cdc-422d-bdef-e4f22b8ed4ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:51.426148 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac9a21-9cdc-422d-bdef-e4f22b8ed4ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:51.430408 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 63/122] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 06:58:51 2022] PUT /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888/users/1eb74ccd914b40dbac0acc5a1d975c76/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:58:51.431277 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c00ee2-1f6a-4d73-bd84-086648822ff9 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:51.432197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-0bac9a21-9cdc-422d-bdef-e4f22b8ed4ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:58:51.432197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:51.432197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:51.432197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:51.432197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:51.432197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:51.432197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:51.432197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:51.432197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:51.432197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:51.432197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:51.432197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:51.432197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:51.432197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:51.432197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:51.432197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:51.433007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:51.433007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:51.433007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:51.433007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:51.433007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:51.433007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:51.433007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:51.433007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:51.433007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:51.433007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:51.433007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:51.433007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:51.433007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:51.433007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:51.433007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:51.433007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:51.433811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:51.433811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:51.433811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:51.433811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:51.433811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:51.433811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:51.433811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:51.433811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:51.433811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:51.433811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:51.433811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:58:51.433811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:58:51.436561 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 60/123] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 06:58:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:58:51.442961 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac9a21-9cdc-422d-bdef-e4f22b8ed4ef None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:51.451019 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:51.621493 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac9a21-9cdc-422d-bdef-e4f22b8ed4ef None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:51.643250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c00ee2-1f6a-4d73-bd84-086648822ff9 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:51.670836 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:51.695146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1e622-1c84-4ad5-bf03-671be334391e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:51.696538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1e622-1c84-4ad5-bf03-671be334391e None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:51.696829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1e622-1c84-4ad5-bf03-671be334391e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:51.697082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1e622-1c84-4ad5-bf03-671be334391e None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:51.697771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1e622-1c84-4ad5-bf03-671be334391e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:51.698072 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1e622-1c84-4ad5-bf03-671be334391e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:51.699501 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1e622-1c84-4ad5-bf03-671be334391e None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:51.708702 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 61/124] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:58:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:51.709025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa3295-934b-4268-9110-7790124854bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:51.709521 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1e622-1c84-4ad5-bf03-671be334391e None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:51.710144 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa3295-934b-4268-9110-7790124854bb None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:51.710619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa3295-934b-4268-9110-7790124854bb None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:51.711334 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa3295-934b-4268-9110-7790124854bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:51.730807 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa3295-934b-4268-9110-7790124854bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:51.738652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-73aa3295-934b-4268-9110-7790124854bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:58:51.738652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:51.738652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:51.738652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:51.738652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:51.738652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:51.738652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:51.738652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:51.738652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:51.738652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:51.738652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:51.738652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:51.738652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:51.738652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:51.738652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:51.738652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:51.739691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:51.739691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:51.739691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:51.739691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:51.739691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:51.739691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:51.739691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:51.739691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:51.739691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:51.739691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:51.739691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:51.739691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:51.739691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:51.739691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:51.739691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:51.739691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:51.740438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:51.740438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:51.740438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:51.740438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:51.740438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:51.740438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:51.740438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:51.740438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:51.740438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:51.740438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:51.740438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:58:51.740438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:58:51.743770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 64/125] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 06:58:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:58:51.744933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa3295-934b-4268-9110-7790124854bb None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:51.818159 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:51.863377 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa3295-934b-4268-9110-7790124854bb None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:51.879195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1e622-1c84-4ad5-bf03-671be334391e None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:51.900603 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c3daa-ce2c-44aa-b7e2-809bdcaf18cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:51.901616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c3daa-ce2c-44aa-b7e2-809bdcaf18cb None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:51.902090 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c3daa-ce2c-44aa-b7e2-809bdcaf18cb None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:51.902558 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c3daa-ce2c-44aa-b7e2-809bdcaf18cb None admin] PATH_INFO: `/v3/users/2ff4f07bfac14f428ece0dfbc5ec3579` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:51.903915 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c3daa-ce2c-44aa-b7e2-809bdcaf18cb None admin] RBAC: Authorizing `identity:get_user(user_id=2ff4f07bfac14f428ece0dfbc5ec3579)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:51.905672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c3daa-ce2c-44aa-b7e2-809bdcaf18cb None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:51.907489 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 65/126] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:58:51 2022] GET /identity/v3/users/2ff4f07bfac14f428ece0dfbc5ec3579 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:51.908540 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c3daa-ce2c-44aa-b7e2-809bdcaf18cb None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:51.945599 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf284679-437a-4646-8164-7930c42fdecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:51.946257 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf284679-437a-4646-8164-7930c42fdecd None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:51.946783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf284679-437a-4646-8164-7930c42fdecd None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:51.947066 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf284679-437a-4646-8164-7930c42fdecd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:51.961303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:51.968693 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf284679-437a-4646-8164-7930c42fdecd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:51.978386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-cf284679-437a-4646-8164-7930c42fdecd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:58:51.978386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:51.978386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:51.978386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:51.978386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:51.978386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:51.978386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:51.978386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:51.978386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:51.978386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:51.978386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:51.978386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:51.978386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:51.978386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:51.978386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:51.978386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:51.979285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:51.979285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:51.979285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:51.979285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:51.979285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:51.979285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:51.979285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:51.979285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:51.979285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:51.979285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:51.979285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:51.979285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:51.979285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:51.979285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:51.979285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:51.979285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:51.980096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:51.980096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:51.980096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:51.980096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:51.980096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:51.980096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:51.980096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:51.980096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:51.980096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:51.980096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:51.980096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:58:51.980096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:58:51.980096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 62/127] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:58:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:58:51.980720 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf284679-437a-4646-8164-7930c42fdecd None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:52.025603 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c3daa-ce2c-44aa-b7e2-809bdcaf18cb None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:52.062793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87728456-bcc2-47c7-b66b-d574304fbde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:52.063896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87728456-bcc2-47c7-b66b-d574304fbde7 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:52.064687 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87728456-bcc2-47c7-b66b-d574304fbde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:52.065154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87728456-bcc2-47c7-b66b-d574304fbde7 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:52.066067 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87728456-bcc2-47c7-b66b-d574304fbde7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:52.066690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87728456-bcc2-47c7-b66b-d574304fbde7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:52.068450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87728456-bcc2-47c7-b66b-d574304fbde7 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:52.077908 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 66/128] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:58:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:52.079044 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87728456-bcc2-47c7-b66b-d574304fbde7 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:52.123884 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf284679-437a-4646-8164-7930c42fdecd None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:52.150216 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:52.161234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d1c40-25c5-4f04-9667-09dc17701c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:52.162230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d1c40-25c5-4f04-9667-09dc17701c53 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:52.162707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d1c40-25c5-4f04-9667-09dc17701c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:52.163176 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d1c40-25c5-4f04-9667-09dc17701c53 None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:52.165614 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d1c40-25c5-4f04-9667-09dc17701c53 None admin] RBAC: Authorizing `identity:get_project(project_id=92ca046a717e4adab11f7371ad3cc888)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:52.167461 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d1c40-25c5-4f04-9667-09dc17701c53 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:52.170641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 63/129] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:58:51 2022] GET /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:52.172143 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d1c40-25c5-4f04-9667-09dc17701c53 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:52.211610 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87728456-bcc2-47c7-b66b-d574304fbde7 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:52.223751 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:52.287321 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de1f0ca-34e6-4631-bbc3-2dcfd7ee88ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:52.287965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1f0ca-34e6-4631-bbc3-2dcfd7ee88ad None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:52.288246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1f0ca-34e6-4631-bbc3-2dcfd7ee88ad None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:52.288525 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1f0ca-34e6-4631-bbc3-2dcfd7ee88ad None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:52.289196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de1f0ca-34e6-4631-bbc3-2dcfd7ee88ad None admin] RBAC: Adding query filter params (name=member) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:52.291549 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de1f0ca-34e6-4631-bbc3-2dcfd7ee88ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:52.292978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de1f0ca-34e6-4631-bbc3-2dcfd7ee88ad None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:52.311917 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 67/130] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:58:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:52.312738 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de1f0ca-34e6-4631-bbc3-2dcfd7ee88ad None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:52.366617 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d1c40-25c5-4f04-9667-09dc17701c53 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:52.410770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b79abb-69ea-4488-81c1-ae768bc65de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:52.411375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b79abb-69ea-4488-81c1-ae768bc65de2 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:52.411687 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b79abb-69ea-4488-81c1-ae768bc65de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:52.411957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b79abb-69ea-4488-81c1-ae768bc65de2 None admin] PATH_INFO: `/v3/users/1eb74ccd914b40dbac0acc5a1d975c76` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:52.413816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b79abb-69ea-4488-81c1-ae768bc65de2 None admin] RBAC: Authorizing `identity:get_user(user_id=1eb74ccd914b40dbac0acc5a1d975c76)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:52.416234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b79abb-69ea-4488-81c1-ae768bc65de2 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:52.417999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 64/131] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:58:52 2022] GET /identity/v3/users/1eb74ccd914b40dbac0acc5a1d975c76 => generated 275 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:52.418595 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b79abb-69ea-4488-81c1-ae768bc65de2 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:52.496826 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de1f0ca-34e6-4631-bbc3-2dcfd7ee88ad None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:52.546794 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:52.565771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f93f1e-aa95-4bbb-a889-ce2b1fd8a99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:52.567512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f93f1e-aa95-4bbb-a889-ce2b1fd8a99d None admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:52.567796 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f93f1e-aa95-4bbb-a889-ce2b1fd8a99d None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:52.568079 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f93f1e-aa95-4bbb-a889-ce2b1fd8a99d None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888/users/2ff4f07bfac14f428ece0dfbc5ec3579/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:52.573846 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f93f1e-aa95-4bbb-a889-ce2b1fd8a99d None admin] RBAC: Authorizing `identity:create_grant(project_id=92ca046a717e4adab11f7371ad3cc888, user_id=2ff4f07bfac14f428ece0dfbc5ec3579, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:52.575360 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f93f1e-aa95-4bbb-a889-ce2b1fd8a99d None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:52.600295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 68/132] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 06:58:52 2022] PUT /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888/users/2ff4f07bfac14f428ece0dfbc5ec3579/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:58:52.601157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f93f1e-aa95-4bbb-a889-ce2b1fd8a99d None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:52.634188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b79abb-69ea-4488-81c1-ae768bc65de2 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:52.675891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:52.711466 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eff545-5f45-467d-908d-7ddd44f0652c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:52.712137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eff545-5f45-467d-908d-7ddd44f0652c None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:52.712442 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eff545-5f45-467d-908d-7ddd44f0652c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:52.712703 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eff545-5f45-467d-908d-7ddd44f0652c None admin] PATH_INFO: `/v3/users/1eb74ccd914b40dbac0acc5a1d975c76` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:52.719623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eff545-5f45-467d-908d-7ddd44f0652c None admin] RBAC: Authorizing `identity:get_user(user_id=1eb74ccd914b40dbac0acc5a1d975c76)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:52.721150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eff545-5f45-467d-908d-7ddd44f0652c None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:52.726320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 65/133] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:58:52 2022] GET /identity/v3/users/1eb74ccd914b40dbac0acc5a1d975c76 => generated 275 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:52.733967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eff545-5f45-467d-908d-7ddd44f0652c None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:52.828776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f93f1e-aa95-4bbb-a889-ce2b1fd8a99d None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:52.872524 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b743528-adea-435f-8b60-72c904497adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:52.873185 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b743528-adea-435f-8b60-72c904497adf None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:52.873494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b743528-adea-435f-8b60-72c904497adf None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:52.873768 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b743528-adea-435f-8b60-72c904497adf None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:52.876044 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b743528-adea-435f-8b60-72c904497adf None admin] RBAC: Authorizing `identity:get_project(project_id=92ca046a717e4adab11f7371ad3cc888)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:52.877926 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b743528-adea-435f-8b60-72c904497adf None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:52.879935 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 69/134] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:58:52 2022] GET /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:52.887210 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b743528-adea-435f-8b60-72c904497adf None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:52.919102 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eff545-5f45-467d-908d-7ddd44f0652c None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:52.975820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51529af2-b5c4-45b3-9999-4158ad4df478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:52.976960 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51529af2-b5c4-45b3-9999-4158ad4df478 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:52.977674 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51529af2-b5c4-45b3-9999-4158ad4df478 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:52.978152 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51529af2-b5c4-45b3-9999-4158ad4df478 None admin] PATH_INFO: `/v3/projects/96f9988774414935ad8f1a88bef57274` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:52.979581 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51529af2-b5c4-45b3-9999-4158ad4df478 None admin] RBAC: Authorizing `identity:get_project(project_id=96f9988774414935ad8f1a88bef57274)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:52.983859 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:52.984269 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51529af2-b5c4-45b3-9999-4158ad4df478 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:52.986485 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 66/135] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:58:52 2022] GET /identity/v3/projects/96f9988774414935ad8f1a88bef57274 => generated 312 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:52.988178 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e408af0-731b-404b-89e4-f2a6dcf6c962 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:52.988752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e408af0-731b-404b-89e4-f2a6dcf6c962 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:52.989234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e408af0-731b-404b-89e4-f2a6dcf6c962 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:52.991398 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 67/136] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:52.992629 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e408af0-731b-404b-89e4-f2a6dcf6c962 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:53.024867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:53.033319 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b743528-adea-435f-8b60-72c904497adf None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:53.074882 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cd66c-82fc-48ff-8391-e7e43583753e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:53.075587 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cd66c-82fc-48ff-8391-e7e43583753e None admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:53.075870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cd66c-82fc-48ff-8391-e7e43583753e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:53.076150 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cd66c-82fc-48ff-8391-e7e43583753e None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888/users/1eb74ccd914b40dbac0acc5a1d975c76/roles/c0ef5f54fa5e4cbb93cb48c9c02b83b7` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:53.078316 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cd66c-82fc-48ff-8391-e7e43583753e None admin] RBAC: Authorizing `identity:create_grant(project_id=92ca046a717e4adab11f7371ad3cc888, user_id=1eb74ccd914b40dbac0acc5a1d975c76, role_id=c0ef5f54fa5e4cbb93cb48c9c02b83b7)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:53.079813 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cd66c-82fc-48ff-8391-e7e43583753e None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:53.099643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 70/137] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 06:58:52 2022] PUT /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888/users/1eb74ccd914b40dbac0acc5a1d975c76/roles/c0ef5f54fa5e4cbb93cb48c9c02b83b7 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 06:58:53.101350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79cc71-731e-4661-ba89-f44d0cda2247 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:53.101639 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79cc71-731e-4661-ba89-f44d0cda2247 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:53.101916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79cc71-731e-4661-ba89-f44d0cda2247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:53.140632 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-1e79cc71-731e-4661-ba89-f44d0cda2247 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:53.141897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-1e79cc71-731e-4661-ba89-f44d0cda2247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:53.199931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e408af0-731b-404b-89e4-f2a6dcf6c962 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:53.258952 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 71/138] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:53.263207 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fe877-569a-4ac4-90f7-db35f79e7676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:53.264155 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fe877-569a-4ac4-90f7-db35f79e7676 None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:53.264659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fe877-569a-4ac4-90f7-db35f79e7676 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:53.265132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fe877-569a-4ac4-90f7-db35f79e7676 None admin] PATH_INFO: `/v3/projects/96f9988774414935ad8f1a88bef57274/users/1eb74ccd914b40dbac0acc5a1d975c76/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:53.267350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1fe877-569a-4ac4-90f7-db35f79e7676 None admin] RBAC: Authorizing `identity:create_grant(project_id=96f9988774414935ad8f1a88bef57274, user_id=1eb74ccd914b40dbac0acc5a1d975c76, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:53.269116 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1fe877-569a-4ac4-90f7-db35f79e7676 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:53.272807 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca5823-a147-4a55-9a64-db8f24ce10d5 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:53.273410 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca5823-a147-4a55-9a64-db8f24ce10d5 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:53.273988 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca5823-a147-4a55-9a64-db8f24ce10d5 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:53.275751 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 72/139] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:58:53.292307 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b01105-cb27-4f8f-8006-dc7d003b8ebd None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:53.293012 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b01105-cb27-4f8f-8006-dc7d003b8ebd None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:53.293510 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b01105-cb27-4f8f-8006-dc7d003b8ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:53.295234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 68/140] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 06:58:52 2022] PUT /identity/v3/projects/96f9988774414935ad8f1a88bef57274/users/1eb74ccd914b40dbac0acc5a1d975c76/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:58:53.332787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-04b01105-cb27-4f8f-8006-dc7d003b8ebd None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:53.334395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-04b01105-cb27-4f8f-8006-dc7d003b8ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:53.457439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 73/141] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:53.646409 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fe877-569a-4ac4-90f7-db35f79e7676 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:53.722729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:53.768227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fe877-569a-4ac4-90f7-db35f79e7676 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:53.810182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485d3cf-a9d2-40dc-b5ea-2c113d9cee03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:53.810827 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485d3cf-a9d2-40dc-b5ea-2c113d9cee03 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:53.811107 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485d3cf-a9d2-40dc-b5ea-2c113d9cee03 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:53.811381 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485d3cf-a9d2-40dc-b5ea-2c113d9cee03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:53.812537 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485d3cf-a9d2-40dc-b5ea-2c113d9cee03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:53.814081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485d3cf-a9d2-40dc-b5ea-2c113d9cee03 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:53.816351 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 69/142] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:58:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:53.822330 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b01105-cb27-4f8f-8006-dc7d003b8ebd None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:53.834410 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485d3cf-a9d2-40dc-b5ea-2c113d9cee03 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:53.852025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:53.969465 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b01105-cb27-4f8f-8006-dc7d003b8ebd None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:54.016972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5dcda4-3371-440e-ba0f-eb47ad04ad86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:54.016972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5dcda4-3371-440e-ba0f-eb47ad04ad86 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:54.016972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5dcda4-3371-440e-ba0f-eb47ad04ad86 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:54.016972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5dcda4-3371-440e-ba0f-eb47ad04ad86 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:54.017388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5dcda4-3371-440e-ba0f-eb47ad04ad86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:54.017388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5dcda4-3371-440e-ba0f-eb47ad04ad86 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:54.052927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 74/143] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 23 06:58:53 2022] POST /identity/v3/users => generated 283 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:58:54.054202 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ff9ee-b29d-47d2-8b43-46f76a568f6e None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:54.054490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ff9ee-b29d-47d2-8b43-46f76a568f6e None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:54.054779 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ff9ee-b29d-47d2-8b43-46f76a568f6e None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:54.059056 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 75/144] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:54.061389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485d3cf-a9d2-40dc-b5ea-2c113d9cee03 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:54.065005 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52bfaa-bb3e-4bbc-a1db-5cd0117d6d82 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:54.065289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52bfaa-bb3e-4bbc-a1db-5cd0117d6d82 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:54.065560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52bfaa-bb3e-4bbc-a1db-5cd0117d6d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:54.104781 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-7e52bfaa-bb3e-4bbc-a1db-5cd0117d6d82 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:54.106023 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-7e52bfaa-bb3e-4bbc-a1db-5cd0117d6d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:54.127816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:54.141760 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b0ffd-def8-4868-9663-7156935607c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:54.142506 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b0ffd-def8-4868-9663-7156935607c2 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:54.142860 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b0ffd-def8-4868-9663-7156935607c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:54.143206 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b0ffd-def8-4868-9663-7156935607c2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:54.166654 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b0ffd-def8-4868-9663-7156935607c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:54.173966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-301b0ffd-def8-4868-9663-7156935607c2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 06:58:54.173966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:54.173966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:54.173966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:54.173966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:54.173966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:54.173966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:54.173966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:54.173966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:54.173966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:54.173966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:54.173966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:54.173966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:54.173966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:54.173966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:54.173966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:54.175050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:54.175050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:54.175050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:54.175050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:54.175050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:54.175050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:54.175050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:54.175050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:54.175050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:54.175050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:54.175050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:54.175050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:54.175050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:54.175050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:54.175050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:54.175050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:54.176006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:54.176006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:54.176006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:54.176006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:54.176006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:54.176006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:54.176006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:54.176006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:54.176006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:54.176006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:54.176006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 06:58:54.176006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:58:54.180498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 70/145] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:58:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:58:54.187004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b0ffd-def8-4868-9663-7156935607c2 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:54.265180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 76/146] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:54.334976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b0ffd-def8-4868-9663-7156935607c2 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:54.375099 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a714f2b-9191-4d24-8c1b-aba0a73b3583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:54.375865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a714f2b-9191-4d24-8c1b-aba0a73b3583 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:54.376212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a714f2b-9191-4d24-8c1b-aba0a73b3583 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:54.376555 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a714f2b-9191-4d24-8c1b-aba0a73b3583 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:54.377327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a714f2b-9191-4d24-8c1b-aba0a73b3583 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:54.377700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a714f2b-9191-4d24-8c1b-aba0a73b3583 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:54.379209 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a714f2b-9191-4d24-8c1b-aba0a73b3583 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:54.389252 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 71/147] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:58:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:54.394875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52bfaa-bb3e-4bbc-a1db-5cd0117d6d82 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:54.397727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:54.518004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52bfaa-bb3e-4bbc-a1db-5cd0117d6d82 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:54.558159 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34549eec-ecf6-4230-bc93-15e19c050bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:54.558863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34549eec-ecf6-4230-bc93-15e19c050bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:54.559147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34549eec-ecf6-4230-bc93-15e19c050bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:54.559441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34549eec-ecf6-4230-bc93-15e19c050bc6 None admin] PATH_INFO: `/v3/users/dbc5ca4b49e54cb7ab4206845a8b8e18` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:54.560607 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34549eec-ecf6-4230-bc93-15e19c050bc6 None admin] RBAC: Authorizing `identity:get_user(user_id=dbc5ca4b49e54cb7ab4206845a8b8e18)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:54.562071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34549eec-ecf6-4230-bc93-15e19c050bc6 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:54.563720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 77/148] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:58:54 2022] GET /identity/v3/users/dbc5ca4b49e54cb7ab4206845a8b8e18 => generated 289 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:54.569606 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a714f2b-9191-4d24-8c1b-aba0a73b3583 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:54.634128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:54.698637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a714f2b-9191-4d24-8c1b-aba0a73b3583 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:54.735187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c384314-9ab8-42c4-9ac5-f981a4531dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:54.735867 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c384314-9ab8-42c4-9ac5-f981a4531dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:54.736145 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c384314-9ab8-42c4-9ac5-f981a4531dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:54.736420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c384314-9ab8-42c4-9ac5-f981a4531dc0 None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:54.737614 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c384314-9ab8-42c4-9ac5-f981a4531dc0 None admin] RBAC: Authorizing `identity:get_project(project_id=92ca046a717e4adab11f7371ad3cc888)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:54.739141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c384314-9ab8-42c4-9ac5-f981a4531dc0 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:54.740816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 72/149] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:58:54 2022] GET /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:54.747232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34549eec-ecf6-4230-bc93-15e19c050bc6 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:54.835845 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:54.871925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c384314-9ab8-42c4-9ac5-f981a4531dc0 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:54.874124 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34549eec-ecf6-4230-bc93-15e19c050bc6 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:54.939596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615ea79-453f-44c7-8c8d-4e0f4cb6c9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:54.940595 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615ea79-453f-44c7-8c8d-4e0f4cb6c9ac None admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:54.940877 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615ea79-453f-44c7-8c8d-4e0f4cb6c9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:54.941153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615ea79-453f-44c7-8c8d-4e0f4cb6c9ac None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888/users/dbc5ca4b49e54cb7ab4206845a8b8e18/roles/ff4e7aa3dd7e4272a3cd1cafb32baa24` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:54.948616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c615ea79-453f-44c7-8c8d-4e0f4cb6c9ac None admin] RBAC: Authorizing `identity:create_grant(project_id=92ca046a717e4adab11f7371ad3cc888, user_id=dbc5ca4b49e54cb7ab4206845a8b8e18, role_id=ff4e7aa3dd7e4272a3cd1cafb32baa24)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:54.950097 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c615ea79-453f-44c7-8c8d-4e0f4cb6c9ac None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:54.977293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 78/150] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 06:58:54 2022] PUT /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888/users/dbc5ca4b49e54cb7ab4206845a8b8e18/roles/ff4e7aa3dd7e4272a3cd1cafb32baa24 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 06:58:55.005090 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c384314-9ab8-42c4-9ac5-f981a4531dc0 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:55.035887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:55.057671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9a36d-29dd-4ccd-8825-7dced41fa24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:55.058360 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9a36d-29dd-4ccd-8825-7dced41fa24d None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:55.058646 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9a36d-29dd-4ccd-8825-7dced41fa24d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:55.058923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9a36d-29dd-4ccd-8825-7dced41fa24d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:55.073453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9a36d-29dd-4ccd-8825-7dced41fa24d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:55.078910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-84b9a36d-29dd-4ccd-8825-7dced41fa24d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:58:55.078910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:55.078910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:55.078910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:55.078910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:55.078910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:55.078910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:55.078910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:55.078910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:55.078910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:55.078910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:55.078910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:55.078910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:55.078910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:55.078910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:55.078910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:55.079820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:55.079820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:55.079820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:55.079820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:55.079820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:55.079820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:55.079820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:55.079820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:55.079820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:55.079820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:55.079820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:55.079820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:55.079820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:55.079820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:55.079820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:55.079820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:55.080625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:55.080625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:55.080625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:55.080625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:55.080625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:55.080625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:55.080625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:55.080625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:55.080625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:55.080625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:55.080625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:58:55.080625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:58:55.080625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 73/151] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:58:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:58:55.086389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615ea79-453f-44c7-8c8d-4e0f4cb6c9ac None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:55.210645 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615ea79-453f-44c7-8c8d-4e0f4cb6c9ac None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:55.255828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:55.256762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d5e72-41af-4815-93b0-2b6597c2dedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:55.257405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d5e72-41af-4815-93b0-2b6597c2dedc None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:55.257695 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d5e72-41af-4815-93b0-2b6597c2dedc None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:55.257974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d5e72-41af-4815-93b0-2b6597c2dedc None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:55.258670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d5e72-41af-4815-93b0-2b6597c2dedc None admin] RBAC: Adding query filter params (name=member) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:55.259283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d5e72-41af-4815-93b0-2b6597c2dedc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:55.260782 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d5e72-41af-4815-93b0-2b6597c2dedc None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:55.274751 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 79/152] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:58:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:55.284803 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9a36d-29dd-4ccd-8825-7dced41fa24d None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:55.422285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9a36d-29dd-4ccd-8825-7dced41fa24d None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:55.462488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aab712-8951-4fac-bfab-8c101bdac2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:55.463599 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aab712-8951-4fac-bfab-8c101bdac2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:55.464095 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aab712-8951-4fac-bfab-8c101bdac2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:55.464566 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aab712-8951-4fac-bfab-8c101bdac2d7 None admin] PATH_INFO: `/v3/users/1eb74ccd914b40dbac0acc5a1d975c76` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:55.465926 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aab712-8951-4fac-bfab-8c101bdac2d7 None admin] RBAC: Authorizing `identity:get_user(user_id=1eb74ccd914b40dbac0acc5a1d975c76)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:55.467631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aab712-8951-4fac-bfab-8c101bdac2d7 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:55.469459 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 74/153] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:58:55 2022] GET /identity/v3/users/1eb74ccd914b40dbac0acc5a1d975c76 => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:55.475500 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d5e72-41af-4815-93b0-2b6597c2dedc None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:55.597584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7026758-97c7-4276-9809-02c2736d1d13 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:55.598177 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7026758-97c7-4276-9809-02c2736d1d13 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:55.598669 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7026758-97c7-4276-9809-02c2736d1d13 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:55.600807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 75/154] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:58:55.606045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906973f-97f7-439c-b1e0-646000ad96f2 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:55.606992 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906973f-97f7-439c-b1e0-646000ad96f2 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:55.607502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906973f-97f7-439c-b1e0-646000ad96f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:55.628718 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d5e72-41af-4815-93b0-2b6597c2dedc None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:55.642683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-a906973f-97f7-439c-b1e0-646000ad96f2 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:55.644244 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-a906973f-97f7-439c-b1e0-646000ad96f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:55.713563 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:55.742352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 76/155] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:55.743520 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4d704-6fdc-4b45-9b30-c082bf3dc93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:55.744250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4d704-6fdc-4b45-9b30-c082bf3dc93b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:55.744608 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4d704-6fdc-4b45-9b30-c082bf3dc93b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:55.744960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4d704-6fdc-4b45-9b30-c082bf3dc93b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:55.819586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:55.831381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4d704-6fdc-4b45-9b30-c082bf3dc93b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:55.832894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4d704-6fdc-4b45-9b30-c082bf3dc93b None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:55.948688 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 80/156] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:58:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:58:55.954555 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906973f-97f7-439c-b1e0-646000ad96f2 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:56.074275 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfb4f4-41e0-41a4-b7a1-0b7548d27156 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:56.074455 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfb4f4-41e0-41a4-b7a1-0b7548d27156 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:56.074736 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfb4f4-41e0-41a4-b7a1-0b7548d27156 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:56.076166 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 81/157] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:58:56.081946 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed67607-e5d1-4ef5-b515-a86933d851da None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:56.082231 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed67607-e5d1-4ef5-b515-a86933d851da None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:56.082511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed67607-e5d1-4ef5-b515-a86933d851da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:56.092824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906973f-97f7-439c-b1e0-646000ad96f2 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:56.124464 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-3ed67607-e5d1-4ef5-b515-a86933d851da None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:56.125662 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3ed67607-e5d1-4ef5-b515-a86933d851da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:56.129136 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f62788-db25-4045-a97f-33fc66efda30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:56.130065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f62788-db25-4045-a97f-33fc66efda30 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:56.130535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f62788-db25-4045-a97f-33fc66efda30 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:56.130996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f62788-db25-4045-a97f-33fc66efda30 None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:56.132370 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f62788-db25-4045-a97f-33fc66efda30 None admin] RBAC: Authorizing `identity:get_project(project_id=92ca046a717e4adab11f7371ad3cc888)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:56.134193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f62788-db25-4045-a97f-33fc66efda30 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:56.136364 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 77/158] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:58:55 2022] GET /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:56.141952 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f62788-db25-4045-a97f-33fc66efda30 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:56.194478 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:56.241875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 82/159] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:56.242950 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bd0b1-a0a3-4dc2-9d96-5383a71832ba None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:56.243237 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bd0b1-a0a3-4dc2-9d96-5383a71832ba None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:56.243574 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bd0b1-a0a3-4dc2-9d96-5383a71832ba None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:56.244849 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 83/160] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:56.257047 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d5156-2cdd-4653-9142-bc64cfec6d18 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:56.257403 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d5156-2cdd-4653-9142-bc64cfec6d18 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:56.257599 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d5156-2cdd-4653-9142-bc64cfec6d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:56.259497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f62788-db25-4045-a97f-33fc66efda30 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:56.297018 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab517e-2ce0-4c10-b272-128b0ef110a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:56.298051 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab517e-2ce0-4c10-b272-128b0ef110a9 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:56.298513 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab517e-2ce0-4c10-b272-128b0ef110a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:56.299032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab517e-2ce0-4c10-b272-128b0ef110a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:56.300043 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab517e-2ce0-4c10-b272-128b0ef110a9 None admin] RBAC: Adding query filter params (role.id=7670c7c450b9404ba57e12743cb6c458, scope.project.id=92ca046a717e4adab11f7371ad3cc888, user.id=1eb74ccd914b40dbac0acc5a1d975c76) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:56.300439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-7f3d5156-2cdd-4653-9142-bc64cfec6d18 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:56.300790 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab517e-2ce0-4c10-b272-128b0ef110a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:56.301656 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-7f3d5156-2cdd-4653-9142-bc64cfec6d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:56.302682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab517e-2ce0-4c10-b272-128b0ef110a9 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:56.310027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 78/161] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:58:56 2022] GET /identity/v3/role_assignments?user.id=1eb74ccd914b40dbac0acc5a1d975c76&scope.project.id=92ca046a717e4adab11f7371ad3cc888&role.id=7670c7c450b9404ba57e12743cb6c458 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:58:56.326239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:56.398802 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 84/162] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:58:56.519574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab517e-2ce0-4c10-b272-128b0ef110a9 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:56.637575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab517e-2ce0-4c10-b272-128b0ef110a9 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:56.645400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:56.678154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c3ef8-8f52-4ecc-91d9-eba61d612f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:56.678794 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c3ef8-8f52-4ecc-91d9-eba61d612f4f None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:56.679130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c3ef8-8f52-4ecc-91d9-eba61d612f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:56.679483 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c3ef8-8f52-4ecc-91d9-eba61d612f4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:56.695267 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c3ef8-8f52-4ecc-91d9-eba61d612f4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:56.704312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-0a6c3ef8-8f52-4ecc-91d9-eba61d612f4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:58:56.704312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:56.704312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:56.704312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:56.704312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:56.704312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:56.704312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:56.704312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:56.704312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:56.704312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:56.704312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:56.704312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:56.704312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:56.704312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:56.704312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:56.704312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:56.705268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:56.705268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:56.705268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:56.705268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:56.705268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:56.705268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:56.705268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:56.705268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:56.705268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:56.705268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:56.705268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:56.705268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:56.705268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:56.705268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:56.705268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:56.705268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:56.706065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:56.706065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:56.706065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:56.706065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:56.706065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:56.706065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:56.706065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:56.706065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:56.706065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:56.706065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:56.706065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:58:56.706065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:58:56.706065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 79/163] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 06:58:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:58:56.708669 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d5156-2cdd-4653-9142-bc64cfec6d18 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:56.733445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c3ef8-8f52-4ecc-91d9-eba61d612f4f None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:56.866044 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d5156-2cdd-4653-9142-bc64cfec6d18 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:56.887934 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c3ef8-8f52-4ecc-91d9-eba61d612f4f None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:56.928722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f68499-f615-4c5e-8bc0-169479e18be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:56.929731 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f68499-f615-4c5e-8bc0-169479e18be5 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:56.930206 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f68499-f615-4c5e-8bc0-169479e18be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:56.930703 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f68499-f615-4c5e-8bc0-169479e18be5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:56.936824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c27f04-c5ce-4300-926c-a34d707f5840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:56.937543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c27f04-c5ce-4300-926c-a34d707f5840 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:56.937825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c27f04-c5ce-4300-926c-a34d707f5840 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:56.938097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c27f04-c5ce-4300-926c-a34d707f5840 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:56.938778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c27f04-c5ce-4300-926c-a34d707f5840 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:56.939081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c27f04-c5ce-4300-926c-a34d707f5840 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:56.941007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c27f04-c5ce-4300-926c-a34d707f5840 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:56.950623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 80/164] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:58:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:56.956413 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f68499-f615-4c5e-8bc0-169479e18be5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:56.959622 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c27f04-c5ce-4300-926c-a34d707f5840 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:56.967082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-61f68499-f615-4c5e-8bc0-169479e18be5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:58:56.967082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:56.967082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:56.967082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:56.967082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:56.967082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:56.967082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:56.967082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:56.967082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:56.967082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:56.967082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:56.967082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:56.967082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:56.967082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:56.967082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:56.967082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:56.968108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:56.968108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:56.968108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:56.968108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:56.968108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:56.968108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:56.968108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:56.968108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:56.968108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:56.968108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:56.968108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:56.968108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:56.968108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:56.968108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:56.968108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:56.968108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:56.969078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:56.969078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:56.969078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:56.969078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:56.969078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:56.969078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:56.969078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:56.969078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:56.969078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:56.969078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:56.969078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:58:56.969078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:58:56.971611 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 85/165] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:58:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:58:56.972755 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f68499-f615-4c5e-8bc0-169479e18be5 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:57.009470 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:57.031781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:57.147124 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f68499-f615-4c5e-8bc0-169479e18be5 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:57.161262 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c27f04-c5ce-4300-926c-a34d707f5840 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:57.204952 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d129b5f-4e70-4aa0-812c-8a06c417243a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:57.204952 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d129b5f-4e70-4aa0-812c-8a06c417243a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:57.204952 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d129b5f-4e70-4aa0-812c-8a06c417243a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:57.204952 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d129b5f-4e70-4aa0-812c-8a06c417243a None admin] PATH_INFO: `/v3/users/2ff4f07bfac14f428ece0dfbc5ec3579` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:57.205336 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d129b5f-4e70-4aa0-812c-8a06c417243a None admin] RBAC: Authorizing `identity:get_user(user_id=2ff4f07bfac14f428ece0dfbc5ec3579)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:57.205336 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d129b5f-4e70-4aa0-812c-8a06c417243a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:57.205483 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b199998-b87a-4847-ab2c-3d4e3e786c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:57.205569 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 86/166] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:58:56 2022] GET /identity/v3/users/2ff4f07bfac14f428ece0dfbc5ec3579 => generated 247 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:57.205569 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d129b5f-4e70-4aa0-812c-8a06c417243a None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:57.209004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b199998-b87a-4847-ab2c-3d4e3e786c78 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:57.209305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b199998-b87a-4847-ab2c-3d4e3e786c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:57.209589 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b199998-b87a-4847-ab2c-3d4e3e786c78 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:57.221601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b199998-b87a-4847-ab2c-3d4e3e786c78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:57.226819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-0b199998-b87a-4847-ab2c-3d4e3e786c78 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:58:57.226819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:57.226819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:57.226819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:57.226819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:57.226819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:57.226819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:57.226819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:57.226819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:57.226819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:57.226819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:57.226819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:57.226819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:57.226819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:57.226819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:57.226819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:57.227774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:57.227774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:57.227774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:57.227774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:57.227774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:57.227774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:57.227774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:57.227774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:57.227774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:57.227774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:57.227774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:57.227774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:57.227774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:57.227774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:57.227774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:57.227774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:57.229279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:57.229279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:57.229279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:57.229279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:57.229279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:57.229279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:57.229279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:57.229279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:57.229279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:57.229279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:57.229279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:58:57.229279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:58:57.229279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 81/167] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 06:58:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:58:57.230028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b199998-b87a-4847-ab2c-3d4e3e786c78 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:57.317608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:57.334889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b199998-b87a-4847-ab2c-3d4e3e786c78 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:57.335840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:57.355134 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d129b5f-4e70-4aa0-812c-8a06c417243a None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:57.401895 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87336c6-6cf5-4ce7-a248-33287871acb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:57.402644 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87336c6-6cf5-4ce7-a248-33287871acb4 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:57.403024 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87336c6-6cf5-4ce7-a248-33287871acb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:57.403366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87336c6-6cf5-4ce7-a248-33287871acb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:57.432924 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad5da94-85e5-48b8-99f2-180148b0911a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:57.433611 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5da94-85e5-48b8-99f2-180148b0911a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:57.433898 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5da94-85e5-48b8-99f2-180148b0911a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:57.434198 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5da94-85e5-48b8-99f2-180148b0911a None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:57.434949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad5da94-85e5-48b8-99f2-180148b0911a None admin] RBAC: Adding query filter params (name=member) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:57.435263 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad5da94-85e5-48b8-99f2-180148b0911a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:57.441177 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad5da94-85e5-48b8-99f2-180148b0911a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:57.459764 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 87/168] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:58:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:57.464401 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad5da94-85e5-48b8-99f2-180148b0911a None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:57.500032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87336c6-6cf5-4ce7-a248-33287871acb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:57.501517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87336c6-6cf5-4ce7-a248-33287871acb4 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:57.603828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:57.624955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 82/169] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:58:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:58:57.626836 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87336c6-6cf5-4ce7-a248-33287871acb4 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:57.649435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad5da94-85e5-48b8-99f2-180148b0911a None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:57.734897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b526e-9d5c-48b3-87fa-f1b0098d96b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:57.735550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b526e-9d5c-48b3-87fa-f1b0098d96b3 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:57.735824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b526e-9d5c-48b3-87fa-f1b0098d96b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:57.736265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b526e-9d5c-48b3-87fa-f1b0098d96b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:57.737077 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b526e-9d5c-48b3-87fa-f1b0098d96b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:57.737523 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b526e-9d5c-48b3-87fa-f1b0098d96b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:57.739065 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b526e-9d5c-48b3-87fa-f1b0098d96b3 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:57.756638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 88/170] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:58:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:57.759918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b526e-9d5c-48b3-87fa-f1b0098d96b3 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:57.775895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:57.831105 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87336c6-6cf5-4ce7-a248-33287871acb4 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:57.904166 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33fe93-18cd-4e92-b15f-ec75385c8cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:57.904820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33fe93-18cd-4e92-b15f-ec75385c8cee None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:57.905095 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33fe93-18cd-4e92-b15f-ec75385c8cee None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:57.905432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33fe93-18cd-4e92-b15f-ec75385c8cee None admin] PATH_INFO: `/v3/users/1eb74ccd914b40dbac0acc5a1d975c76` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:57.911445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33fe93-18cd-4e92-b15f-ec75385c8cee None admin] RBAC: Authorizing `identity:get_user(user_id=1eb74ccd914b40dbac0acc5a1d975c76)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:57.912944 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33fe93-18cd-4e92-b15f-ec75385c8cee None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:57.914621 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 83/171] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:58:57 2022] GET /identity/v3/users/1eb74ccd914b40dbac0acc5a1d975c76 => generated 275 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:57.922256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33fe93-18cd-4e92-b15f-ec75385c8cee None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:57.983762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b526e-9d5c-48b3-87fa-f1b0098d96b3 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:58.037245 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:58.054501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23441885-15ca-4fd8-9ab3-ac2831a9ac1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:58.055145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23441885-15ca-4fd8-9ab3-ac2831a9ac1a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:58.055432 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23441885-15ca-4fd8-9ab3-ac2831a9ac1a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:58.055718 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23441885-15ca-4fd8-9ab3-ac2831a9ac1a None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:58.059140 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23441885-15ca-4fd8-9ab3-ac2831a9ac1a None admin] RBAC: Authorizing `identity:get_project(project_id=92ca046a717e4adab11f7371ad3cc888)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:58.068829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23441885-15ca-4fd8-9ab3-ac2831a9ac1a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:58.070565 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 89/172] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:58:57 2022] GET /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888 => generated 298 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:58.074470 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23441885-15ca-4fd8-9ab3-ac2831a9ac1a None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:58.137641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33fe93-18cd-4e92-b15f-ec75385c8cee None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:58.205481 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0514d6b-9f65-4be2-b727-3563648848f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:58.206031 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0514d6b-9f65-4be2-b727-3563648848f4 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:58.206308 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0514d6b-9f65-4be2-b727-3563648848f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:58.206621 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0514d6b-9f65-4be2-b727-3563648848f4 None admin] PATH_INFO: `/v3/users/1eb74ccd914b40dbac0acc5a1d975c76` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:58.208249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0514d6b-9f65-4be2-b727-3563648848f4 None admin] RBAC: Authorizing `identity:get_user(user_id=1eb74ccd914b40dbac0acc5a1d975c76)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:58.209729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0514d6b-9f65-4be2-b727-3563648848f4 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:58.211333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 84/173] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:58:57 2022] GET /identity/v3/users/1eb74ccd914b40dbac0acc5a1d975c76 => generated 275 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:58.216151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0514d6b-9f65-4be2-b727-3563648848f4 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:58.222293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:58.279770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23441885-15ca-4fd8-9ab3-ac2831a9ac1a None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:58.354089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d30050-7f7e-4367-ba00-d7cbd1b0100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:58.354915 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d30050-7f7e-4367-ba00-d7cbd1b0100b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:58.355192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d30050-7f7e-4367-ba00-d7cbd1b0100b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:58.355477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d30050-7f7e-4367-ba00-d7cbd1b0100b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:58.417712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0514d6b-9f65-4be2-b727-3563648848f4 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:58.471625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:58.472667 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d30050-7f7e-4367-ba00-d7cbd1b0100b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:58.474297 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d30050-7f7e-4367-ba00-d7cbd1b0100b None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:58.489693 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196bb03-08fd-4d4c-b26a-12c9d02d71d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:58.490429 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196bb03-08fd-4d4c-b26a-12c9d02d71d8 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:58.490710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196bb03-08fd-4d4c-b26a-12c9d02d71d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:58.491009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196bb03-08fd-4d4c-b26a-12c9d02d71d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:58.491955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196bb03-08fd-4d4c-b26a-12c9d02d71d8 None admin] RBAC: Adding query filter params (role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.project.id=92ca046a717e4adab11f7371ad3cc888, user.id=2ff4f07bfac14f428ece0dfbc5ec3579) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:58.492268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196bb03-08fd-4d4c-b26a-12c9d02d71d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:58.493846 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196bb03-08fd-4d4c-b26a-12c9d02d71d8 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:58.500537 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 85/174] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:58:58 2022] GET /identity/v3/role_assignments?user.id=2ff4f07bfac14f428ece0dfbc5ec3579&scope.project.id=92ca046a717e4adab11f7371ad3cc888&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:58.501395 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196bb03-08fd-4d4c-b26a-12c9d02d71d8 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:58.589590 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:58.628338 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196bb03-08fd-4d4c-b26a-12c9d02d71d8 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:58.638937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 90/175] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:58:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:58:58.641034 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5a13c-20ef-4b1f-8828-bf7d2de905e2 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:58.641324 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5a13c-20ef-4b1f-8828-bf7d2de905e2 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:58.641603 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5a13c-20ef-4b1f-8828-bf7d2de905e2 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:58.643189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 91/176] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:58.644231 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2928a1-d898-4ce4-9899-be5bab013b01 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:58.644515 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2928a1-d898-4ce4-9899-be5bab013b01 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:58.644790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2928a1-d898-4ce4-9899-be5bab013b01 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:58.645998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 92/177] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:58.646946 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2928a1-d898-4ce4-9899-be5bab013b01 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:58.677553 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb5800-0734-4ae6-9928-bc652bae0560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:58.678185 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb5800-0734-4ae6-9928-bc652bae0560 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:58.678487 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb5800-0734-4ae6-9928-bc652bae0560 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:58.679000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb5800-0734-4ae6-9928-bc652bae0560 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:58.755124 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:58.771717 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb5800-0734-4ae6-9928-bc652bae0560 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:58.774710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb5800-0734-4ae6-9928-bc652bae0560 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:58.781395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2928a1-d898-4ce4-9899-be5bab013b01 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:58.825671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666f45f-3305-4bd7-9d72-b65dfd60d5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:58.827837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666f45f-3305-4bd7-9d72-b65dfd60d5ab None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:58.828553 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666f45f-3305-4bd7-9d72-b65dfd60d5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:58.829812 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666f45f-3305-4bd7-9d72-b65dfd60d5ab None admin] PATH_INFO: `/v3/projects/96f9988774414935ad8f1a88bef57274` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:58.832350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666f45f-3305-4bd7-9d72-b65dfd60d5ab None admin] RBAC: Authorizing `identity:get_project(project_id=96f9988774414935ad8f1a88bef57274)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:58.836170 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666f45f-3305-4bd7-9d72-b65dfd60d5ab None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:58.838994 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 93/178] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:58:58 2022] GET /identity/v3/projects/96f9988774414935ad8f1a88bef57274 => generated 312 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:58.840666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df96f7a-a55d-4c6c-bc21-bc0fa87bb6fa None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:58.841152 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df96f7a-a55d-4c6c-bc21-bc0fa87bb6fa None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:58.841624 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df96f7a-a55d-4c6c-bc21-bc0fa87bb6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:58.864982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-3df96f7a-a55d-4c6c-bc21-bc0fa87bb6fa None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:58.867259 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3df96f7a-a55d-4c6c-bc21-bc0fa87bb6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:58.904804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 86/179] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:58:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:58:58.907265 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b736d-2691-4a7b-8424-b3bb4539a6b0 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:58.907847 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b736d-2691-4a7b-8424-b3bb4539a6b0 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:58.908267 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b736d-2691-4a7b-8424-b3bb4539a6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:58.915203 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:58.934711 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-c42b736d-2691-4a7b-8424-b3bb4539a6b0 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:58.935955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c42b736d-2691-4a7b-8424-b3bb4539a6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:58.962030 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 94/180] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:58.967688 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df96f7a-a55d-4c6c-bc21-bc0fa87bb6fa None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:59.010412 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 87/181] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:59.011195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b736d-2691-4a7b-8424-b3bb4539a6b0 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:59.035304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:59.077240 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df96f7a-a55d-4c6c-bc21-bc0fa87bb6fa None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:59.116457 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece92ac0-e9e8-4ce6-9fb2-719889eef242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:59.117387 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece92ac0-e9e8-4ce6-9fb2-719889eef242 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:59.117815 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece92ac0-e9e8-4ce6-9fb2-719889eef242 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:59.118310 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece92ac0-e9e8-4ce6-9fb2-719889eef242 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:59.119341 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece92ac0-e9e8-4ce6-9fb2-719889eef242 None admin] RBAC: Adding query filter params (role.id=7670c7c450b9404ba57e12743cb6c458, scope.project.id=96f9988774414935ad8f1a88bef57274, user.id=1eb74ccd914b40dbac0acc5a1d975c76) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:59.120073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece92ac0-e9e8-4ce6-9fb2-719889eef242 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:59.121863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece92ac0-e9e8-4ce6-9fb2-719889eef242 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:59.128585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 95/182] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:58:58 2022] GET /identity/v3/role_assignments?user.id=1eb74ccd914b40dbac0acc5a1d975c76&scope.project.id=96f9988774414935ad8f1a88bef57274&role.id=7670c7c450b9404ba57e12743cb6c458 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:59.129963 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edfb04-7beb-49b2-8324-697f3b6289e9 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:59.130391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edfb04-7beb-49b2-8324-697f3b6289e9 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:59.130830 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edfb04-7beb-49b2-8324-697f3b6289e9 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:59.130941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b736d-2691-4a7b-8424-b3bb4539a6b0 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:59.132574 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 96/183] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:59.133949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ae752-38b5-429a-ba32-a8c239f30c49 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:59.134381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ae752-38b5-429a-ba32-a8c239f30c49 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:59.134915 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ae752-38b5-429a-ba32-a8c239f30c49 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:59.136566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 97/184] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:58:59.138608 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd10cbb-15a4-4895-8150-d586956380f7 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:59.139052 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd10cbb-15a4-4895-8150-d586956380f7 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:59.139502 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd10cbb-15a4-4895-8150-d586956380f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:59.161388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-edd10cbb-15a4-4895-8150-d586956380f7 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:59.163111 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-edd10cbb-15a4-4895-8150-d586956380f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:59.175932 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6a9b1-b8be-47ad-9c5d-538f680ca540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:59.176595 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6a9b1-b8be-47ad-9c5d-538f680ca540 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:59.176873 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6a9b1-b8be-47ad-9c5d-538f680ca540 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:59.177145 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6a9b1-b8be-47ad-9c5d-538f680ca540 None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:59.178308 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6a9b1-b8be-47ad-9c5d-538f680ca540 None admin] RBAC: Authorizing `identity:get_project(project_id=92ca046a717e4adab11f7371ad3cc888)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:59.180456 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6a9b1-b8be-47ad-9c5d-538f680ca540 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:59.182802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 88/185] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:58:59 2022] GET /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:59.191438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7417374-656f-4a52-b27a-91f1ffc580d8 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:59.191438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7417374-656f-4a52-b27a-91f1ffc580d8 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:59.191438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7417374-656f-4a52-b27a-91f1ffc580d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:59.223915 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:59.224886 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-e7417374-656f-4a52-b27a-91f1ffc580d8 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:58:59.225868 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e7417374-656f-4a52-b27a-91f1ffc580d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:58:59.279442 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 98/186] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:59.280274 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd10cbb-15a4-4895-8150-d586956380f7 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:59.330755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 89/187] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:58:59.331656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7417374-656f-4a52-b27a-91f1ffc580d8 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:59.349785 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:59.412088 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd10cbb-15a4-4895-8150-d586956380f7 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:59.442033 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7417374-656f-4a52-b27a-91f1ffc580d8 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:59.455099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910fe45-9793-42ad-8a66-251635122905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:59.456314 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910fe45-9793-42ad-8a66-251635122905 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:59.456811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910fe45-9793-42ad-8a66-251635122905 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:59.457446 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910fe45-9793-42ad-8a66-251635122905 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:59.458786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6910fe45-9793-42ad-8a66-251635122905 None admin] RBAC: Adding query filter params (role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7, scope.project.id=92ca046a717e4adab11f7371ad3cc888, user.id=1eb74ccd914b40dbac0acc5a1d975c76) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:59.459684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6910fe45-9793-42ad-8a66-251635122905 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:59.461778 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6910fe45-9793-42ad-8a66-251635122905 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:59.469870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 99/188] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:58:59 2022] GET /identity/v3/role_assignments?user.id=1eb74ccd914b40dbac0acc5a1d975c76&scope.project.id=92ca046a717e4adab11f7371ad3cc888&role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:59.471169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910fe45-9793-42ad-8a66-251635122905 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:59.503653 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a36dbd-2932-4566-8594-cd55d59a8da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:59.504685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a36dbd-2932-4566-8594-cd55d59a8da9 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:59.505163 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a36dbd-2932-4566-8594-cd55d59a8da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:59.506220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a36dbd-2932-4566-8594-cd55d59a8da9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:59.531087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a36dbd-2932-4566-8594-cd55d59a8da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:59.536449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-d5a36dbd-2932-4566-8594-cd55d59a8da9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 06:58:59.536449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:59.536449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:59.536449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:59.536449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:59.536449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:59.536449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:59.536449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:59.536449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:59.536449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:59.536449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:59.536449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:59.536449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:59.536449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:59.536449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:59.536449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:59.537402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:59.537402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:59.537402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:59.537402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:59.537402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:59.537402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:59.537402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:59.537402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:59.537402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:59.537402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:59.537402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:59.537402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:59.537402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:59.537402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:59.537402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:59.537402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:59.538203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:59.538203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:59.538203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:59.538203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:59.538203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:59.538203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:59.538203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:59.538203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:59.538203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:59.538203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:59.538203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 06:58:59.538203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:58:59.544830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 90/189] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:58:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:58:59.551303 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a36dbd-2932-4566-8594-cd55d59a8da9 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:59.552786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:59.651156 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910fe45-9793-42ad-8a66-251635122905 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:59.667895 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:59.697047 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c5f95-c7ae-424e-80b3-e22e59e801ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:59.697743 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c5f95-c7ae-424e-80b3-e22e59e801ab None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:59.698066 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c5f95-c7ae-424e-80b3-e22e59e801ab None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:59.698473 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c5f95-c7ae-424e-80b3-e22e59e801ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:59.700192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a36dbd-2932-4566-8594-cd55d59a8da9 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:59.714454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c5f95-c7ae-424e-80b3-e22e59e801ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:59.720286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-005c5f95-c7ae-424e-80b3-e22e59e801ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:58:59.720286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:59.720286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:59.720286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:59.720286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:59.720286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:59.720286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:59.720286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:59.720286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:59.720286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:59.720286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:59.720286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:59.720286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:59.720286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:59.720286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:59.720286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:59.721189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:59.721189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:59.721189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:59.721189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:59.721189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:59.721189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:59.721189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:59.721189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:59.721189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:59.721189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:59.721189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:59.721189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:59.721189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:59.721189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:59.721189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:59.721189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:59.721960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:59.721960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:59.721960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:59.721960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:59.721960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:59.721960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:59.721960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:59.721960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:59.721960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:59.721960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:59.721960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:58:59.721960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:58:59.721960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 100/190] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 06:58:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:58:59.722598 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c5f95-c7ae-424e-80b3-e22e59e801ab None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:59.749584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca756abf-3318-4938-af95-611082bb0014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:59.750196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca756abf-3318-4938-af95-611082bb0014 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:59.750470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca756abf-3318-4938-af95-611082bb0014 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:59.750740 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca756abf-3318-4938-af95-611082bb0014 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:59.751406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca756abf-3318-4938-af95-611082bb0014 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:58:59.751724 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca756abf-3318-4938-af95-611082bb0014 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:59.753132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca756abf-3318-4938-af95-611082bb0014 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:59.761105 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 91/191] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:58:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:59.761875 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca756abf-3318-4938-af95-611082bb0014 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:59.824821 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:59.839022 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c5f95-c7ae-424e-80b3-e22e59e801ab None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:59.872045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca756abf-3318-4938-af95-611082bb0014 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:58:59.877403 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836ee49-6f3d-4365-bed9-14200b6fc2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:59.878038 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836ee49-6f3d-4365-bed9-14200b6fc2bd None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:59.878311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836ee49-6f3d-4365-bed9-14200b6fc2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:59.878578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836ee49-6f3d-4365-bed9-14200b6fc2bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:59.879745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836ee49-6f3d-4365-bed9-14200b6fc2bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:58:59.881214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836ee49-6f3d-4365-bed9-14200b6fc2bd None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:58:59.882804 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 101/192] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:58:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:58:59.883591 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836ee49-6f3d-4365-bed9-14200b6fc2bd None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:58:59.908185 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b392c64-03b9-48f4-8598-f3ad19d0750c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:58:59.908806 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b392c64-03b9-48f4-8598-f3ad19d0750c None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:58:59.909082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b392c64-03b9-48f4-8598-f3ad19d0750c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:58:59.909391 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b392c64-03b9-48f4-8598-f3ad19d0750c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:58:59.916762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:58:59.921621 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b392c64-03b9-48f4-8598-f3ad19d0750c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:58:59.926601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-7b392c64-03b9-48f4-8598-f3ad19d0750c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:58:59.926601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:58:59.926601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:58:59.926601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:58:59.926601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:58:59.926601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:58:59.926601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:58:59.926601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:58:59.926601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:58:59.926601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:58:59.926601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:58:59.926601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:58:59.926601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:58:59.926601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:58:59.926601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:58:59.926601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:58:59.927627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:58:59.927627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:58:59.927627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:58:59.927627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:58:59.927627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:58:59.927627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:58:59.927627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:58:59.927627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:58:59.927627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:58:59.927627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:58:59.927627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:58:59.927627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:58:59.927627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:58:59.927627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:58:59.927627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:58:59.927627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:58:59.928439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:58:59.928439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:58:59.928439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:58:59.928439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:58:59.928439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:58:59.928439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:58:59.928439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:58:59.928439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:58:59.928439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:58:59.928439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:58:59.928439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:58:59.928439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:58:59.928439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 92/193] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 06:58:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:58:59.929109 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b392c64-03b9-48f4-8598-f3ad19d0750c None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:00.014506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836ee49-6f3d-4365-bed9-14200b6fc2bd None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:00.044652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b392c64-03b9-48f4-8598-f3ad19d0750c None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:00.070167 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6b5df-ef1e-4364-9769-13dafd8023c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:00.071110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6b5df-ef1e-4364-9769-13dafd8023c8 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:00.071629 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6b5df-ef1e-4364-9769-13dafd8023c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:00.072106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6b5df-ef1e-4364-9769-13dafd8023c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:00.073485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6b5df-ef1e-4364-9769-13dafd8023c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:00.073795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6b5df-ef1e-4364-9769-13dafd8023c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:00.075555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6b5df-ef1e-4364-9769-13dafd8023c8 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:00.085985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 102/194] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:58:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:00.086838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6b5df-ef1e-4364-9769-13dafd8023c8 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:00.090212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206903d-7e27-4cec-8639-81b99f1cc989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:00.090946 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206903d-7e27-4cec-8639-81b99f1cc989 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:00.091326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206903d-7e27-4cec-8639-81b99f1cc989 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:00.091718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206903d-7e27-4cec-8639-81b99f1cc989 None admin] PATH_INFO: `/v3/users/dbc5ca4b49e54cb7ab4206845a8b8e18` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:00.092936 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206903d-7e27-4cec-8639-81b99f1cc989 None admin] RBAC: Authorizing `identity:get_user(user_id=dbc5ca4b49e54cb7ab4206845a8b8e18)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:00.094411 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206903d-7e27-4cec-8639-81b99f1cc989 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:00.096025 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 93/195] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:58:59 2022] GET /identity/v3/users/dbc5ca4b49e54cb7ab4206845a8b8e18 => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:00.096755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206903d-7e27-4cec-8639-81b99f1cc989 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:00.112763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:00.170462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:00.208120 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206903d-7e27-4cec-8639-81b99f1cc989 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:00.211293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6b5df-ef1e-4364-9769-13dafd8023c8 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:00.248572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275728e-0b55-447e-8c8a-f014503417e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:00.249157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590ace5-ff4c-4999-a6e9-0e65f5df8fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:00.249277 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275728e-0b55-447e-8c8a-f014503417e1 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:00.249453 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275728e-0b55-447e-8c8a-f014503417e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:00.249784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590ace5-ff4c-4999-a6e9-0e65f5df8fee None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:00.249933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275728e-0b55-447e-8c8a-f014503417e1 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:00.250066 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590ace5-ff4c-4999-a6e9-0e65f5df8fee None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:00.250342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590ace5-ff4c-4999-a6e9-0e65f5df8fee None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:00.250721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275728e-0b55-447e-8c8a-f014503417e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:00.251023 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590ace5-ff4c-4999-a6e9-0e65f5df8fee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:00.251333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590ace5-ff4c-4999-a6e9-0e65f5df8fee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:00.252770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590ace5-ff4c-4999-a6e9-0e65f5df8fee None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:00.253033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275728e-0b55-447e-8c8a-f014503417e1 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:00.270280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 94/196] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:00.270280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590ace5-ff4c-4999-a6e9-0e65f5df8fee None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:00.284255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 103/197] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 23 06:59:00 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:59:00.292173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275728e-0b55-447e-8c8a-f014503417e1 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:00.389237 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:00.412574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590ace5-ff4c-4999-a6e9-0e65f5df8fee None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:00.424856 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275728e-0b55-447e-8c8a-f014503417e1 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:00.449295 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:00.460440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f6358-80ee-4b74-bc3b-bece01c8b9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:00.461081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f6358-80ee-4b74-bc3b-bece01c8b9ea None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:00.461384 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f6358-80ee-4b74-bc3b-bece01c8b9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:00.461631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f6358-80ee-4b74-bc3b-bece01c8b9ea None admin] PATH_INFO: `/v3/users/033859049778475bb121842de4286c8a` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:00.474711 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f3b98-08ce-4286-9b69-21b10fe3129c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:00.475887 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f3b98-08ce-4286-9b69-21b10fe3129c None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:00.476173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f3b98-08ce-4286-9b69-21b10fe3129c None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:00.476453 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f3b98-08ce-4286-9b69-21b10fe3129c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:00.485756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f6358-80ee-4b74-bc3b-bece01c8b9ea None admin] RBAC: Authorizing `identity:get_user(user_id=033859049778475bb121842de4286c8a)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:00.487971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f6358-80ee-4b74-bc3b-bece01c8b9ea None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:00.494285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 95/198] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:59:00 2022] GET /identity/v3/users/033859049778475bb121842de4286c8a => generated 283 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:00.495696 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f6358-80ee-4b74-bc3b-bece01c8b9ea None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:00.574938 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f3b98-08ce-4286-9b69-21b10fe3129c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:00.576532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f3b98-08ce-4286-9b69-21b10fe3129c None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:00.645462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f6358-80ee-4b74-bc3b-bece01c8b9ea None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:00.693261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:00.720605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8cb2ea-d421-4ea7-82f7-4bbe4a01db3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:00.721721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8cb2ea-d421-4ea7-82f7-4bbe4a01db3d None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:00.722221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8cb2ea-d421-4ea7-82f7-4bbe4a01db3d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:00.722688 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8cb2ea-d421-4ea7-82f7-4bbe4a01db3d None admin] PATH_INFO: `/v3/users/033859049778475bb121842de4286c8a` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:00.724217 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8cb2ea-d421-4ea7-82f7-4bbe4a01db3d None admin] RBAC: Authorizing `identity:get_user(user_id=033859049778475bb121842de4286c8a)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:00.725889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8cb2ea-d421-4ea7-82f7-4bbe4a01db3d None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:00.727768 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 96/199] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:59:00 2022] GET /identity/v3/users/033859049778475bb121842de4286c8a => generated 283 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:00.728884 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8cb2ea-d421-4ea7-82f7-4bbe4a01db3d None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:00.734010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 104/200] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:00.739272 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f3b98-08ce-4286-9b69-21b10fe3129c None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:00.806765 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:00.857543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8cb2ea-d421-4ea7-82f7-4bbe4a01db3d None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:00.862392 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f3b98-08ce-4286-9b69-21b10fe3129c None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:00.894241 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f45473-b43c-40fb-ab99-7eaef1e0ca9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:00.899393 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eac447-97e9-4b32-9fab-1fae0cddb7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:00.899393 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eac447-97e9-4b32-9fab-1fae0cddb7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:00.899736 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eac447-97e9-4b32-9fab-1fae0cddb7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:00.899857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eac447-97e9-4b32-9fab-1fae0cddb7b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:00.904052 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f45473-b43c-40fb-ab99-7eaef1e0ca9a None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:00.904612 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f45473-b43c-40fb-ab99-7eaef1e0ca9a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:00.905080 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f45473-b43c-40fb-ab99-7eaef1e0ca9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:00.946308 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:00.973912 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eac447-97e9-4b32-9fab-1fae0cddb7b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:00.975377 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eac447-97e9-4b32-9fab-1fae0cddb7b5 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:00.999443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f45473-b43c-40fb-ab99-7eaef1e0ca9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:00.999443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f45473-b43c-40fb-ab99-7eaef1e0ca9a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:01.056639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:01.114307 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 97/202] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:01.114504 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 105/202] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:01.115377 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eac447-97e9-4b32-9fab-1fae0cddb7b5 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:01.115563 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f45473-b43c-40fb-ab99-7eaef1e0ca9a None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:01.222480 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:01.234274 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f45473-b43c-40fb-ab99-7eaef1e0ca9a None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:01.251245 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eac447-97e9-4b32-9fab-1fae0cddb7b5 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:01.280431 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532e0e8-ce65-4dc8-a3b7-0465957f4aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:01.281466 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532e0e8-ce65-4dc8-a3b7-0465957f4aff None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:01.281750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532e0e8-ce65-4dc8-a3b7-0465957f4aff None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:01.282021 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532e0e8-ce65-4dc8-a3b7-0465957f4aff None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:01.283592 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532e0e8-ce65-4dc8-a3b7-0465957f4aff None admin] RBAC: Authorizing `identity:get_project(project_id=92ca046a717e4adab11f7371ad3cc888)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:01.285451 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532e0e8-ce65-4dc8-a3b7-0465957f4aff None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:01.286239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d0d9c-c6c8-49b1-85d4-aca08bbac97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:01.286932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d0d9c-c6c8-49b1-85d4-aca08bbac97e None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:01.287270 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d0d9c-c6c8-49b1-85d4-aca08bbac97e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:01.287581 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 98/203] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:01 2022] GET /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:01.287682 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d0d9c-c6c8-49b1-85d4-aca08bbac97e None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:01.288529 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532e0e8-ce65-4dc8-a3b7-0465957f4aff None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:01.288987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d0d9c-c6c8-49b1-85d4-aca08bbac97e None admin] RBAC: Authorizing `identity:get_project(project_id=e594e988024e47b4a48b43d8f828c605)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:01.290545 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d0d9c-c6c8-49b1-85d4-aca08bbac97e None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:01.292341 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 106/204] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:01 2022] GET /identity/v3/projects/e594e988024e47b4a48b43d8f828c605 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:01.293200 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d0d9c-c6c8-49b1-85d4-aca08bbac97e None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:01.318604 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:01.403021 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532e0e8-ce65-4dc8-a3b7-0465957f4aff None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:01.424488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d0d9c-c6c8-49b1-85d4-aca08bbac97e None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:01.451367 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4465a95-d0c3-4608-b5fd-49f28d2879f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:01.452214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4465a95-d0c3-4608-b5fd-49f28d2879f8 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:01.452586 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4465a95-d0c3-4608-b5fd-49f28d2879f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:01.452970 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4465a95-d0c3-4608-b5fd-49f28d2879f8 None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:01.454189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4465a95-d0c3-4608-b5fd-49f28d2879f8 None admin] RBAC: Authorizing `identity:get_project(project_id=e594e988024e47b4a48b43d8f828c605)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:01.456233 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4465a95-d0c3-4608-b5fd-49f28d2879f8 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:01.468915 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f992f-8565-4c0f-86d1-779058d45600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:01.469842 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f992f-8565-4c0f-86d1-779058d45600 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:01.470320 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f992f-8565-4c0f-86d1-779058d45600 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:01.470775 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f992f-8565-4c0f-86d1-779058d45600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:01.471856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 99/205] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:01 2022] GET /identity/v3/projects/e594e988024e47b4a48b43d8f828c605 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:01.472451 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f992f-8565-4c0f-86d1-779058d45600 None admin] RBAC: Adding query filter params (role.id=ff4e7aa3dd7e4272a3cd1cafb32baa24, scope.project.id=92ca046a717e4adab11f7371ad3cc888, user.id=dbc5ca4b49e54cb7ab4206845a8b8e18) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:01.472846 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4465a95-d0c3-4608-b5fd-49f28d2879f8 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:01.473314 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f992f-8565-4c0f-86d1-779058d45600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:01.474948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f992f-8565-4c0f-86d1-779058d45600 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:01.482281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 107/206] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:59:01 2022] GET /identity/v3/role_assignments?user.id=dbc5ca4b49e54cb7ab4206845a8b8e18&scope.project.id=92ca046a717e4adab11f7371ad3cc888&role.id=ff4e7aa3dd7e4272a3cd1cafb32baa24 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:01.483517 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f992f-8565-4c0f-86d1-779058d45600 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:01.503917 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:01.662059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:01.677400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4465a95-d0c3-4608-b5fd-49f28d2879f8 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:01.689743 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f992f-8565-4c0f-86d1-779058d45600 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:01.716073 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3462eef-9210-4583-986b-16f40b4a6649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:01.716688 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3462eef-9210-4583-986b-16f40b4a6649 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:01.716963 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3462eef-9210-4583-986b-16f40b4a6649 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:01.717232 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3462eef-9210-4583-986b-16f40b4a6649 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:01.717967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3462eef-9210-4583-986b-16f40b4a6649 None admin] RBAC: Adding query filter params (role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.project.id=e594e988024e47b4a48b43d8f828c605, user.id=033859049778475bb121842de4286c8a) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:01.718275 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3462eef-9210-4583-986b-16f40b4a6649 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:01.720490 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3462eef-9210-4583-986b-16f40b4a6649 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:01.726197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 100/207] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:59:01 2022] GET /identity/v3/role_assignments?user.id=033859049778475bb121842de4286c8a&scope.project.id=e594e988024e47b4a48b43d8f828c605&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:01.730768 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4121bf-5be3-4cc6-9649-0cbd1763e48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:01.731394 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4121bf-5be3-4cc6-9649-0cbd1763e48a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:01.731698 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4121bf-5be3-4cc6-9649-0cbd1763e48a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:01.731966 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4121bf-5be3-4cc6-9649-0cbd1763e48a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:01.732782 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4121bf-5be3-4cc6-9649-0cbd1763e48a None admin] RBAC: Adding query filter params (role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7, scope.project.id=e594e988024e47b4a48b43d8f828c605, user.id=033859049778475bb121842de4286c8a) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:01.733094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4121bf-5be3-4cc6-9649-0cbd1763e48a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:01.734585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4121bf-5be3-4cc6-9649-0cbd1763e48a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:01.741645 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 108/208] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:59:01 2022] GET /identity/v3/role_assignments?user.id=033859049778475bb121842de4286c8a&scope.project.id=e594e988024e47b4a48b43d8f828c605&role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:02.445849 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84d985-93a0-427a-b1ae-43bfbbd3601f None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:02.446404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84d985-93a0-427a-b1ae-43bfbbd3601f None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:02.446687 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84d985-93a0-427a-b1ae-43bfbbd3601f None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:02.448701 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 101/209] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:02.454127 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50740555-2c3f-4ac8-9163-725f0be36481 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:02.454544 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50740555-2c3f-4ac8-9163-725f0be36481 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:02.454860 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50740555-2c3f-4ac8-9163-725f0be36481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:02.474510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c293a48-bf98-4d92-8c88-ba2603b0be40 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:02.474800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c293a48-bf98-4d92-8c88-ba2603b0be40 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:02.475052 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c293a48-bf98-4d92-8c88-ba2603b0be40 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:02.476173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-50740555-2c3f-4ac8-9163-725f0be36481 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:02.476565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 102/210] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:59:02.477970 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-50740555-2c3f-4ac8-9163-725f0be36481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:02.485608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d362c65-7de6-482f-a772-6a52588077db None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:02.485963 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d362c65-7de6-482f-a772-6a52588077db None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:02.486308 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d362c65-7de6-482f-a772-6a52588077db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:02.505911 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-6d362c65-7de6-482f-a772-6a52588077db None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:02.507440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-6d362c65-7de6-482f-a772-6a52588077db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:02.551156 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:02.557016 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 109/211] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:02.580792 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 103/212] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:02.819622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50740555-2c3f-4ac8-9163-725f0be36481 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:02.846926 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d362c65-7de6-482f-a772-6a52588077db None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:02.931308 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50740555-2c3f-4ac8-9163-725f0be36481 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:02.953403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:02.970732 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925fd80-a983-48fb-9096-921d8d4c6d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:02.971712 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925fd80-a983-48fb-9096-921d8d4c6d80 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:02.972184 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925fd80-a983-48fb-9096-921d8d4c6d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:02.972653 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d362c65-7de6-482f-a772-6a52588077db None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:02.972849 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925fd80-a983-48fb-9096-921d8d4c6d80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:02.984152 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925fd80-a983-48fb-9096-921d8d4c6d80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:02.989477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-7925fd80-a983-48fb-9096-921d8d4c6d80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:02.989477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:02.989477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:02.989477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:02.989477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:02.989477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:02.989477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:02.989477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:02.989477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:02.989477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:02.989477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:02.989477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:02.989477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:02.989477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:02.989477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:02.989477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:02.990357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:02.990357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:02.990357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:02.990357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:02.990357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:02.990357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:02.990357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:02.990357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:02.990357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:02.990357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:02.990357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:02.990357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:02.990357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:02.990357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:02.990357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:02.990357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:02.991183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:02.991183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:02.991183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:02.991183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:02.991183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:02.991183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:02.991183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:02.991183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:02.991183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:02.991183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:02.991183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:02.991183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:02.993878 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 110/213] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:03.000339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925fd80-a983-48fb-9096-921d8d4c6d80 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:03.018561 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16fab5-2adb-4490-a634-aa02116c43d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:03.019566 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16fab5-2adb-4490-a634-aa02116c43d5 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:03.020048 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16fab5-2adb-4490-a634-aa02116c43d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:03.020556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16fab5-2adb-4490-a634-aa02116c43d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:03.021867 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16fab5-2adb-4490-a634-aa02116c43d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:03.023670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16fab5-2adb-4490-a634-aa02116c43d5 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:03.025458 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 104/214] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:03.034814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16fab5-2adb-4490-a634-aa02116c43d5 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:03.064605 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:03.109065 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925fd80-a983-48fb-9096-921d8d4c6d80 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:03.146817 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37d49d-feb1-4901-a68b-0c4d626a2e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:03.147518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37d49d-feb1-4901-a68b-0c4d626a2e8d None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:03.147931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37d49d-feb1-4901-a68b-0c4d626a2e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:03.148255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37d49d-feb1-4901-a68b-0c4d626a2e8d None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:03.148927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37d49d-feb1-4901-a68b-0c4d626a2e8d None admin] RBAC: Adding query filter params (name=member) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:03.149233 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37d49d-feb1-4901-a68b-0c4d626a2e8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:03.150984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37d49d-feb1-4901-a68b-0c4d626a2e8d None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:03.160226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 111/215] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:59:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:03.161143 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16fab5-2adb-4490-a634-aa02116c43d5 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:03.165951 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37d49d-feb1-4901-a68b-0c4d626a2e8d None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:03.203462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490be215-c2a3-492b-9bf3-989839df39d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:03.204527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490be215-c2a3-492b-9bf3-989839df39d1 None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:03.205715 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490be215-c2a3-492b-9bf3-989839df39d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:03.206167 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490be215-c2a3-492b-9bf3-989839df39d1 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:03.206997 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490be215-c2a3-492b-9bf3-989839df39d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:03.208671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490be215-c2a3-492b-9bf3-989839df39d1 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:03.238348 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 105/216] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 23 06:59:03 2022] POST /identity/v3/users => generated 297 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:59:03.281860 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37d49d-feb1-4901-a68b-0c4d626a2e8d None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:03.311804 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:03.322373 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e484bcd-fae9-42aa-bbb0-040e21918575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:03.323032 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e484bcd-fae9-42aa-bbb0-040e21918575 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:03.323881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e484bcd-fae9-42aa-bbb0-040e21918575 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:03.324203 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e484bcd-fae9-42aa-bbb0-040e21918575 None admin] PATH_INFO: `/v3/users/344282022c0845cba868724d0ce84528` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:03.340170 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e484bcd-fae9-42aa-bbb0-040e21918575 None admin] RBAC: Authorizing `identity:get_user(user_id=344282022c0845cba868724d0ce84528)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:03.341740 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e484bcd-fae9-42aa-bbb0-040e21918575 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:03.343487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 112/217] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:59:03 2022] GET /identity/v3/users/344282022c0845cba868724d0ce84528 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:03.349607 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490be215-c2a3-492b-9bf3-989839df39d1 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:03.362594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:03.464966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490be215-c2a3-492b-9bf3-989839df39d1 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:03.499963 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247389a7-f091-4274-af15-223f5e8e9582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:03.500586 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247389a7-f091-4274-af15-223f5e8e9582 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:03.500862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247389a7-f091-4274-af15-223f5e8e9582 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:03.501132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247389a7-f091-4274-af15-223f5e8e9582 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:03.578259 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247389a7-f091-4274-af15-223f5e8e9582 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:03.580094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247389a7-f091-4274-af15-223f5e8e9582 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:03.602258 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:03.699721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 106/218] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:03.706242 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e484bcd-fae9-42aa-bbb0-040e21918575 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:03.770186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2be3c4-def8-4ec7-8393-82544319d082 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:03.770531 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2be3c4-def8-4ec7-8393-82544319d082 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:03.770740 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2be3c4-def8-4ec7-8393-82544319d082 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:03.772405 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 107/219] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:59:03.779312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9924968-7dc2-499b-b63f-43d72e4943a3 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:03.779641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9924968-7dc2-499b-b63f-43d72e4943a3 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:03.779920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9924968-7dc2-499b-b63f-43d72e4943a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:03.803931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-c9924968-7dc2-499b-b63f-43d72e4943a3 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:03.805810 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c9924968-7dc2-499b-b63f-43d72e4943a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:03.832709 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e484bcd-fae9-42aa-bbb0-040e21918575 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:03.875466 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c96cd-ff3f-4287-8f9b-e99797e0e5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:03.876118 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c96cd-ff3f-4287-8f9b-e99797e0e5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:03.876449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c96cd-ff3f-4287-8f9b-e99797e0e5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:03.876725 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c96cd-ff3f-4287-8f9b-e99797e0e5f8 None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:03.877873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c96cd-ff3f-4287-8f9b-e99797e0e5f8 None admin] RBAC: Authorizing `identity:get_project(project_id=e594e988024e47b4a48b43d8f828c605)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:03.879332 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c96cd-ff3f-4287-8f9b-e99797e0e5f8 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:03.881219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 113/220] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:03 2022] GET /identity/v3/projects/e594e988024e47b4a48b43d8f828c605 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:03.882204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6e3ee-eeb2-4e4e-acfe-3d0651fdb6be None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:03.882482 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6e3ee-eeb2-4e4e-acfe-3d0651fdb6be None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:03.882754 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6e3ee-eeb2-4e4e-acfe-3d0651fdb6be None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:03.884244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 114/221] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:03.889165 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6e3ee-eeb2-4e4e-acfe-3d0651fdb6be None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:03.897111 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 108/222] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:59:03.898455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928a746-2cb5-4499-98ab-6f5a8568d9c7 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:03.898804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928a746-2cb5-4499-98ab-6f5a8568d9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:03.899144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928a746-2cb5-4499-98ab-6f5a8568d9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:03.934669 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-0928a746-2cb5-4499-98ab-6f5a8568d9c7 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:03.936474 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-0928a746-2cb5-4499-98ab-6f5a8568d9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:03.985093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:04.004172 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:04.050931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 109/223] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:04.056859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6e3ee-eeb2-4e4e-acfe-3d0651fdb6be None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:04.096202 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18608b-d7a0-4d6a-83ce-942421bd2d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:04.096465 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18608b-d7a0-4d6a-83ce-942421bd2d34 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:04.096745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18608b-d7a0-4d6a-83ce-942421bd2d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:04.097015 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18608b-d7a0-4d6a-83ce-942421bd2d34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:04.098196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18608b-d7a0-4d6a-83ce-942421bd2d34 None admin] RBAC: Adding query filter params (role.id=7670c7c450b9404ba57e12743cb6c458, scope.project.id=e594e988024e47b4a48b43d8f828c605, user.id=344282022c0845cba868724d0ce84528) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:04.098196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18608b-d7a0-4d6a-83ce-942421bd2d34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:04.099591 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18608b-d7a0-4d6a-83ce-942421bd2d34 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:04.105755 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 115/224] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:59:03 2022] GET /identity/v3/role_assignments?user.id=344282022c0845cba868724d0ce84528&scope.project.id=e594e988024e47b4a48b43d8f828c605&role.id=7670c7c450b9404ba57e12743cb6c458 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:04.288144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928a746-2cb5-4499-98ab-6f5a8568d9c7 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:04.406857 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928a746-2cb5-4499-98ab-6f5a8568d9c7 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:04.441342 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18608b-d7a0-4d6a-83ce-942421bd2d34 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:04.448894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4955e-5aa8-4246-8236-a94a20a6e8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:04.449542 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4955e-5aa8-4246-8236-a94a20a6e8c8 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:04.449816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4955e-5aa8-4246-8236-a94a20a6e8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:04.450086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4955e-5aa8-4246-8236-a94a20a6e8c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:04.460574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4955e-5aa8-4246-8236-a94a20a6e8c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:04.465507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-ceb4955e-5aa8-4246-8236-a94a20a6e8c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:59:04.465507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:04.465507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:04.465507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:04.465507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:04.465507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:04.465507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:04.465507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:04.465507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:04.465507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:04.465507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:04.465507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:04.465507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:04.465507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:04.465507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:04.465507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:04.468594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:04.468594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:04.468594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:04.468594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:04.468594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:04.468594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:04.468594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:04.468594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:04.468594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:04.468594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:04.468594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:04.468594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:04.468594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:04.468594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:04.468594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:04.468594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:04.470523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:04.470523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:04.470523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:04.470523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:04.470523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:04.470523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:04.470523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:04.470523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:04.470523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:04.470523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:04.470523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:59:04.470523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:04.470523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 110/225] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 06:59:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:04.475435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4955e-5aa8-4246-8236-a94a20a6e8c8 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:04.531793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:04.603896 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4955e-5aa8-4246-8236-a94a20a6e8c8 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:04.640350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9d468-13b6-4629-90ee-53accaf7abce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:04.640967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9d468-13b6-4629-90ee-53accaf7abce None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:04.641241 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9d468-13b6-4629-90ee-53accaf7abce None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:04.641513 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9d468-13b6-4629-90ee-53accaf7abce None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:04.642181 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9d468-13b6-4629-90ee-53accaf7abce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:04.642480 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9d468-13b6-4629-90ee-53accaf7abce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:04.644002 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9d468-13b6-4629-90ee-53accaf7abce None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:04.657890 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 111/226] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:04.671982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18608b-d7a0-4d6a-83ce-942421bd2d34 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:04.672530 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9d468-13b6-4629-90ee-53accaf7abce None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:04.702174 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:04.767875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a54c6-f742-41df-bc61-20426dc91104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:04.768596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a54c6-f742-41df-bc61-20426dc91104 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:04.768851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a54c6-f742-41df-bc61-20426dc91104 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:04.769059 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a54c6-f742-41df-bc61-20426dc91104 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:04.773070 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:04.784490 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9d468-13b6-4629-90ee-53accaf7abce None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:04.789785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a54c6-f742-41df-bc61-20426dc91104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:04.804039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-3f7a54c6-f742-41df-bc61-20426dc91104 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:59:04.804039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:04.804039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:04.804039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:04.804039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:04.804039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:04.804039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:04.804039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:04.804039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:04.804039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:04.804039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:04.804039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:04.804039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:04.804039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:04.804039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:04.804039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:04.804981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:04.804981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:04.804981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:04.804981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:04.804981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:04.804981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:04.804981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:04.804981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:04.804981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:04.804981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:04.804981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:04.804981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:04.804981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:04.804981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:04.804981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:04.804981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:04.805762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:04.805762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:04.805762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:04.805762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:04.805762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:04.805762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:04.805762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:04.805762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:04.805762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:04.805762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:04.805762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:59:04.805762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:04.805762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 116/227] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 06:59:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:04.811388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a54c6-f742-41df-bc61-20426dc91104 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:04.819651 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeefb14-0246-4beb-93b9-72001d9b4b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:04.820298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeefb14-0246-4beb-93b9-72001d9b4b86 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:04.820576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeefb14-0246-4beb-93b9-72001d9b4b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:04.820849 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeefb14-0246-4beb-93b9-72001d9b4b86 None admin] PATH_INFO: `/v3/users/033859049778475bb121842de4286c8a` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:04.821998 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeefb14-0246-4beb-93b9-72001d9b4b86 None admin] RBAC: Authorizing `identity:get_user(user_id=033859049778475bb121842de4286c8a)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:04.823499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeefb14-0246-4beb-93b9-72001d9b4b86 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:04.825306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 112/228] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:59:04 2022] GET /identity/v3/users/033859049778475bb121842de4286c8a => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:04.830904 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeefb14-0246-4beb-93b9-72001d9b4b86 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:04.930992 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeefb14-0246-4beb-93b9-72001d9b4b86 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:04.965018 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab50984-d739-41e8-a5a5-1da9b2c33fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:04.965656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab50984-d739-41e8-a5a5-1da9b2c33fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:04.965931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab50984-d739-41e8-a5a5-1da9b2c33fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:04.966203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab50984-d739-41e8-a5a5-1da9b2c33fd2 None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:04.967315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab50984-d739-41e8-a5a5-1da9b2c33fd2 None admin] RBAC: Authorizing `identity:get_project(project_id=e594e988024e47b4a48b43d8f828c605)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:04.968808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab50984-d739-41e8-a5a5-1da9b2c33fd2 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:04.970598 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 113/229] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:04 2022] GET /identity/v3/projects/e594e988024e47b4a48b43d8f828c605 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:04.976810 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab50984-d739-41e8-a5a5-1da9b2c33fd2 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:04.991219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a54c6-f742-41df-bc61-20426dc91104 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:05.029740 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:05.051396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd7114-bb64-45fb-ab84-55301041281f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:05.052038 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd7114-bb64-45fb-ab84-55301041281f None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:05.052322 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd7114-bb64-45fb-ab84-55301041281f None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:05.052591 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd7114-bb64-45fb-ab84-55301041281f None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:05.053258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd7114-bb64-45fb-ab84-55301041281f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:05.053554 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd7114-bb64-45fb-ab84-55301041281f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:05.054945 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd7114-bb64-45fb-ab84-55301041281f None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:05.073526 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 117/230] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:05.079216 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd7114-bb64-45fb-ab84-55301041281f None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:05.117105 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab50984-d739-41e8-a5a5-1da9b2c33fd2 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:05.151317 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ce1ef-d17e-496c-b81e-9aa57fe6587d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:05.152130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ce1ef-d17e-496c-b81e-9aa57fe6587d None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:05.152412 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ce1ef-d17e-496c-b81e-9aa57fe6587d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:05.152685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ce1ef-d17e-496c-b81e-9aa57fe6587d None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605/users/033859049778475bb121842de4286c8a/roles/c0ef5f54fa5e4cbb93cb48c9c02b83b7` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:05.155059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ce1ef-d17e-496c-b81e-9aa57fe6587d None admin] RBAC: Authorizing `identity:create_grant(project_id=e594e988024e47b4a48b43d8f828c605, user_id=033859049778475bb121842de4286c8a, role_id=c0ef5f54fa5e4cbb93cb48c9c02b83b7)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:05.159446 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ce1ef-d17e-496c-b81e-9aa57fe6587d None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:05.165607 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:05.175193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 114/231] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 06:59:04 2022] PUT /identity/v3/projects/e594e988024e47b4a48b43d8f828c605/users/033859049778475bb121842de4286c8a/roles/c0ef5f54fa5e4cbb93cb48c9c02b83b7 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 06:59:05.302650 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd7114-bb64-45fb-ab84-55301041281f None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:05.338122 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b0be6-3a3f-4dbe-bf84-8b83c4868d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:05.338771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b0be6-3a3f-4dbe-bf84-8b83c4868d0e None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:05.339044 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b0be6-3a3f-4dbe-bf84-8b83c4868d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:05.339644 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b0be6-3a3f-4dbe-bf84-8b83c4868d0e None admin] PATH_INFO: `/v3/users/033859049778475bb121842de4286c8a` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:05.341836 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b0be6-3a3f-4dbe-bf84-8b83c4868d0e None admin] RBAC: Authorizing `identity:get_user(user_id=033859049778475bb121842de4286c8a)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:05.343341 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b0be6-3a3f-4dbe-bf84-8b83c4868d0e None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:05.345503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 118/232] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:59:05 2022] GET /identity/v3/users/033859049778475bb121842de4286c8a => generated 283 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:05.350994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ce1ef-d17e-496c-b81e-9aa57fe6587d None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:05.488289 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:05.509369 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ce1ef-d17e-496c-b81e-9aa57fe6587d None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:05.557422 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a94d9f0-d4fe-4b38-9344-62fa3a6efd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:05.558100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94d9f0-d4fe-4b38-9344-62fa3a6efd46 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:05.558391 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94d9f0-d4fe-4b38-9344-62fa3a6efd46 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:05.558664 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94d9f0-d4fe-4b38-9344-62fa3a6efd46 None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:05.564139 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a94d9f0-d4fe-4b38-9344-62fa3a6efd46 None admin] RBAC: Authorizing `identity:get_project(project_id=e594e988024e47b4a48b43d8f828c605)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:05.565634 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a94d9f0-d4fe-4b38-9344-62fa3a6efd46 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:05.567550 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 115/233] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:05 2022] GET /identity/v3/projects/e594e988024e47b4a48b43d8f828c605 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:05.573460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b0be6-3a3f-4dbe-bf84-8b83c4868d0e None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:05.723152 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b0be6-3a3f-4dbe-bf84-8b83c4868d0e None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:05.724812 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:05.760032 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914c1b9-1441-48d4-b1a3-aacb182cbc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:05.760909 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914c1b9-1441-48d4-b1a3-aacb182cbc29 None admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:05.761226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914c1b9-1441-48d4-b1a3-aacb182cbc29 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:05.761705 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914c1b9-1441-48d4-b1a3-aacb182cbc29 None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605/users/033859049778475bb121842de4286c8a/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:05.764328 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914c1b9-1441-48d4-b1a3-aacb182cbc29 None admin] RBAC: Authorizing `identity:create_grant(project_id=e594e988024e47b4a48b43d8f828c605, user_id=033859049778475bb121842de4286c8a, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:05.765803 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914c1b9-1441-48d4-b1a3-aacb182cbc29 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:05.786910 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 119/234] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 06:59:05 2022] PUT /identity/v3/projects/e594e988024e47b4a48b43d8f828c605/users/033859049778475bb121842de4286c8a/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:59:05.807745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925a4a4-d268-4267-82b2-c3f4aea2c07f None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:05.807989 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925a4a4-d268-4267-82b2-c3f4aea2c07f None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:05.808625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925a4a4-d268-4267-82b2-c3f4aea2c07f None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:05.810016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 116/235] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:05.826247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cbe4f-dd49-42c4-a9e5-02e572437f09 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:05.826584 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cbe4f-dd49-42c4-a9e5-02e572437f09 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:05.826808 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cbe4f-dd49-42c4-a9e5-02e572437f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:05.853184 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-de8cbe4f-dd49-42c4-a9e5-02e572437f09 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:05.854463 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-de8cbe4f-dd49-42c4-a9e5-02e572437f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:05.951622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 120/236] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:05.992113 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a3041-7917-40a8-91db-8c6b2a8558dc None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:05.992394 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a3041-7917-40a8-91db-8c6b2a8558dc None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:05.992669 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a3041-7917-40a8-91db-8c6b2a8558dc None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:05.994218 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 117/237] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:06.000861 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad598a99-de5d-44f2-becb-da86432e730b None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:06.001149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad598a99-de5d-44f2-becb-da86432e730b None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:06.001424 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad598a99-de5d-44f2-becb-da86432e730b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:06.024221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-ad598a99-de5d-44f2-becb-da86432e730b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:06.025479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ad598a99-de5d-44f2-becb-da86432e730b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:06.113955 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 121/238] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:06.238969 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a3041-7917-40a8-91db-8c6b2a8558dc None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:06.369479 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a3041-7917-40a8-91db-8c6b2a8558dc None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:06.411087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf19fe4-36db-4376-82c9-8c7308a6210a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:06.411757 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf19fe4-36db-4376-82c9-8c7308a6210a None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:06.412229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf19fe4-36db-4376-82c9-8c7308a6210a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:06.412516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf19fe4-36db-4376-82c9-8c7308a6210a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:06.413720 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf19fe4-36db-4376-82c9-8c7308a6210a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:06.415190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf19fe4-36db-4376-82c9-8c7308a6210a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:06.417160 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 118/239] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:06.422554 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad598a99-de5d-44f2-becb-da86432e730b None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:06.454522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:06.469710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf19fe4-36db-4376-82c9-8c7308a6210a None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:06.507817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:06.576321 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad598a99-de5d-44f2-becb-da86432e730b None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:06.598857 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf19fe4-36db-4376-82c9-8c7308a6210a None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:06.617074 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2c9e5-93b7-41c4-b31c-84bd622c54a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:06.617701 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2c9e5-93b7-41c4-b31c-84bd622c54a8 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:06.617982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2c9e5-93b7-41c4-b31c-84bd622c54a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:06.618254 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2c9e5-93b7-41c4-b31c-84bd622c54a8 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:06.618925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd2c9e5-93b7-41c4-b31c-84bd622c54a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:06.621308 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd2c9e5-93b7-41c4-b31c-84bd622c54a8 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:06.634608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ef0f8-b16d-4e24-9201-490d438bb2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:06.635595 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ef0f8-b16d-4e24-9201-490d438bb2f9 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:06.636065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ef0f8-b16d-4e24-9201-490d438bb2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:06.636706 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ef0f8-b16d-4e24-9201-490d438bb2f9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:06.648661 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 122/240] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 23 06:59:06 2022] POST /identity/v3/users => generated 293 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:59:06.648933 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ef0f8-b16d-4e24-9201-490d438bb2f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:06.650191 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbb7e8-1b15-40ad-804c-1103cb868b99 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:06.650620 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbb7e8-1b15-40ad-804c-1103cb868b99 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:06.651028 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbb7e8-1b15-40ad-804c-1103cb868b99 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:06.657059 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 123/241] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:06.658738 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937dd4f-62be-42da-87b1-63463131dbc3 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:06.659185 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937dd4f-62be-42da-87b1-63463131dbc3 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:06.659627 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937dd4f-62be-42da-87b1-63463131dbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:06.662366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-df8ef0f8-b16d-4e24-9201-490d438bb2f9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 06:59:06.662366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:06.662366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:06.662366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:06.662366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:06.662366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:06.662366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:06.662366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:06.662366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:06.662366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:06.662366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:06.662366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:06.662366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:06.662366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:06.662366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:06.662366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:06.663511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:06.663511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:06.663511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:06.663511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:06.663511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:06.663511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:06.663511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:06.663511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:06.663511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:06.663511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:06.663511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:06.663511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:06.663511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:06.663511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:06.663511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:06.663511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:06.664319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:06.664319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:06.664319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:06.664319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:06.664319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:06.664319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:06.664319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:06.664319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:06.664319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:06.664319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:06.664319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 06:59:06.664319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:06.664935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 119/242] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:06.671857 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ef0f8-b16d-4e24-9201-490d438bb2f9 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:06.684985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-4937dd4f-62be-42da-87b1-63463131dbc3 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:06.686513 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4937dd4f-62be-42da-87b1-63463131dbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:06.764555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 124/243] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:06.801270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:06.827689 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ef0f8-b16d-4e24-9201-490d438bb2f9 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:06.880031 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad9427-84c7-4e0f-a43a-f139080e67f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:06.880659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad9427-84c7-4e0f-a43a-f139080e67f9 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:06.880942 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad9427-84c7-4e0f-a43a-f139080e67f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:06.881219 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad9427-84c7-4e0f-a43a-f139080e67f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:06.881896 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad9427-84c7-4e0f-a43a-f139080e67f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:06.882201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad9427-84c7-4e0f-a43a-f139080e67f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:06.883647 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad9427-84c7-4e0f-a43a-f139080e67f9 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:06.893642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 120/244] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:59:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:06.899235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937dd4f-62be-42da-87b1-63463131dbc3 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:06.910191 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:07.003736 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937dd4f-62be-42da-87b1-63463131dbc3 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:07.039002 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f9406-5265-49ec-8fc1-8c457830b76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:07.039684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f9406-5265-49ec-8fc1-8c457830b76f None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:07.039974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f9406-5265-49ec-8fc1-8c457830b76f None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:07.040240 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f9406-5265-49ec-8fc1-8c457830b76f None admin] PATH_INFO: `/v3/users/a1c58bd4231345c28b05c6987217d213` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:07.054759 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f9406-5265-49ec-8fc1-8c457830b76f None admin] RBAC: Authorizing `identity:get_user(user_id=a1c58bd4231345c28b05c6987217d213)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:07.056284 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f9406-5265-49ec-8fc1-8c457830b76f None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:07.057986 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 125/245] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:59:06 2022] GET /identity/v3/users/a1c58bd4231345c28b05c6987217d213 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:07.063874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad9427-84c7-4e0f-a43a-f139080e67f9 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:07.168182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f9406-5265-49ec-8fc1-8c457830b76f None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:07.180437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad9427-84c7-4e0f-a43a-f139080e67f9 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:07.219492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d9624-c33e-4226-a92f-a75e8053e600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:07.220486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d9624-c33e-4226-a92f-a75e8053e600 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:07.220954 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d9624-c33e-4226-a92f-a75e8053e600 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:07.221411 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d9624-c33e-4226-a92f-a75e8053e600 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:07.237606 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:07.248939 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:07.299612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f9406-5265-49ec-8fc1-8c457830b76f None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:07.309836 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481d9624-c33e-4226-a92f-a75e8053e600 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:07.311668 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481d9624-c33e-4226-a92f-a75e8053e600 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:07.343947 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723924a-1136-4ef9-b9cb-1f1ae8914afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:07.344589 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723924a-1136-4ef9-b9cb-1f1ae8914afd None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:07.344871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723924a-1136-4ef9-b9cb-1f1ae8914afd None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:07.345148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723924a-1136-4ef9-b9cb-1f1ae8914afd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:07.360091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723924a-1136-4ef9-b9cb-1f1ae8914afd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:07.366715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-6723924a-1136-4ef9-b9cb-1f1ae8914afd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:07.366715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:07.366715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:07.366715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:07.366715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:07.366715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:07.366715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:07.366715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:07.366715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:07.366715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:07.366715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:07.366715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:07.366715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:07.366715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:07.366715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:07.366715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:07.367713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:07.367713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:07.367713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:07.367713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:07.367713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:07.367713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:07.367713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:07.367713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:07.367713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:07.367713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:07.367713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:07.367713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:07.367713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:07.367713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:07.367713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:07.367713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:07.372386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:07.372386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:07.372386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:07.372386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:07.372386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:07.372386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:07.372386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:07.372386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:07.372386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:07.372386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:07.372386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:07.372386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:07.372386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 126/246] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:07.376776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723924a-1136-4ef9-b9cb-1f1ae8914afd None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:07.445790 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 121/247] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:07.447391 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72c46e-da3c-43bb-ac8e-d3728adfc8d0 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:07.448077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72c46e-da3c-43bb-ac8e-d3728adfc8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:07.448538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72c46e-da3c-43bb-ac8e-d3728adfc8d0 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:07.450861 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 122/248] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:07.457296 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1dd5f-c874-44fe-83a4-a9669abddca1 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:07.457846 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1dd5f-c874-44fe-83a4-a9669abddca1 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:07.458330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1dd5f-c874-44fe-83a4-a9669abddca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:07.481148 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-7aa1dd5f-c874-44fe-83a4-a9669abddca1 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:07.482653 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-7aa1dd5f-c874-44fe-83a4-a9669abddca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:07.508952 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723924a-1136-4ef9-b9cb-1f1ae8914afd None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:07.539137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:07.547953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:07.551389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29012e26-d9d9-4c70-8922-c12cad62f640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:07.552028 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29012e26-d9d9-4c70-8922-c12cad62f640 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:07.552305 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29012e26-d9d9-4c70-8922-c12cad62f640 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:07.552572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29012e26-d9d9-4c70-8922-c12cad62f640 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:07.553237 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29012e26-d9d9-4c70-8922-c12cad62f640 None admin] RBAC: Adding query filter params (name=member) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:07.553532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29012e26-d9d9-4c70-8922-c12cad62f640 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:07.554935 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29012e26-d9d9-4c70-8922-c12cad62f640 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:07.566765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 127/249] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:59:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:07.567866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29012e26-d9d9-4c70-8922-c12cad62f640 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:07.584317 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 123/250] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:07.585259 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1dd5f-c874-44fe-83a4-a9669abddca1 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:07.694473 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1dd5f-c874-44fe-83a4-a9669abddca1 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:07.732735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9558f-e771-4ae8-b808-2931178423b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:07.733308 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9558f-e771-4ae8-b808-2931178423b0 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:07.733597 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9558f-e771-4ae8-b808-2931178423b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:07.733880 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9558f-e771-4ae8-b808-2931178423b0 None admin] PATH_INFO: `/v3/users/344282022c0845cba868724d0ce84528` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:07.736700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9558f-e771-4ae8-b808-2931178423b0 None admin] RBAC: Authorizing `identity:get_user(user_id=344282022c0845cba868724d0ce84528)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:07.737479 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9558f-e771-4ae8-b808-2931178423b0 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:07.739318 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 124/251] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:59:07 2022] GET /identity/v3/users/344282022c0845cba868724d0ce84528 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:07.744532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9558f-e771-4ae8-b808-2931178423b0 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:07.803755 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29012e26-d9d9-4c70-8922-c12cad62f640 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:07.803957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:07.841990 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4dc296-2e37-48e4-b169-64cea72c3997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:07.842630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dc296-2e37-48e4-b169-64cea72c3997 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:07.842909 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dc296-2e37-48e4-b169-64cea72c3997 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:07.843185 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dc296-2e37-48e4-b169-64cea72c3997 None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:07.844346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4dc296-2e37-48e4-b169-64cea72c3997 None admin] RBAC: Authorizing `identity:get_project(project_id=e594e988024e47b4a48b43d8f828c605)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:07.845821 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4dc296-2e37-48e4-b169-64cea72c3997 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:07.847556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 128/252] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:07 2022] GET /identity/v3/projects/e594e988024e47b4a48b43d8f828c605 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:07.853471 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4dc296-2e37-48e4-b169-64cea72c3997 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:07.861694 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9558f-e771-4ae8-b808-2931178423b0 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:07.898811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5e85e-3b0c-48e1-ac50-f968aaaed606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:07.900002 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5e85e-3b0c-48e1-ac50-f968aaaed606 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:07.900524 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5e85e-3b0c-48e1-ac50-f968aaaed606 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:07.901044 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5e85e-3b0c-48e1-ac50-f968aaaed606 None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:07.902461 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5e85e-3b0c-48e1-ac50-f968aaaed606 None admin] RBAC: Authorizing `identity:get_project(project_id=e594e988024e47b4a48b43d8f828c605)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:07.904305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5e85e-3b0c-48e1-ac50-f968aaaed606 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:07.906267 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 125/253] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:07 2022] GET /identity/v3/projects/e594e988024e47b4a48b43d8f828c605 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:07.912104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5e85e-3b0c-48e1-ac50-f968aaaed606 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:07.914592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:07.962920 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4dc296-2e37-48e4-b169-64cea72c3997 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:08.024704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5e85e-3b0c-48e1-ac50-f968aaaed606 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:08.047874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec496a5b-380f-41fc-9c77-a95232a7d83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:08.048076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:08.048870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec496a5b-380f-41fc-9c77-a95232a7d83f None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:08.049153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec496a5b-380f-41fc-9c77-a95232a7d83f None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:08.049429 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec496a5b-380f-41fc-9c77-a95232a7d83f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:08.050196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec496a5b-380f-41fc-9c77-a95232a7d83f None admin] RBAC: Adding query filter params (role.id=ff4e7aa3dd7e4272a3cd1cafb32baa24, scope.project.id=e594e988024e47b4a48b43d8f828c605, user.id=a1c58bd4231345c28b05c6987217d213) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:08.050475 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec496a5b-380f-41fc-9c77-a95232a7d83f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:08.052088 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec496a5b-380f-41fc-9c77-a95232a7d83f None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:08.058526 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 129/254] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:59:07 2022] GET /identity/v3/role_assignments?user.id=a1c58bd4231345c28b05c6987217d213&scope.project.id=e594e988024e47b4a48b43d8f828c605&role.id=ff4e7aa3dd7e4272a3cd1cafb32baa24 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:08.059533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec496a5b-380f-41fc-9c77-a95232a7d83f None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:08.077015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e9fe3-1042-43e6-8d8b-f7c83bd26523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:08.077711 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e9fe3-1042-43e6-8d8b-f7c83bd26523 None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:08.079589 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e9fe3-1042-43e6-8d8b-f7c83bd26523 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:08.079886 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e9fe3-1042-43e6-8d8b-f7c83bd26523 None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605/users/344282022c0845cba868724d0ce84528/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:08.083818 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e9fe3-1042-43e6-8d8b-f7c83bd26523 None admin] RBAC: Authorizing `identity:create_grant(project_id=e594e988024e47b4a48b43d8f828c605, user_id=344282022c0845cba868724d0ce84528, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:08.087315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e9fe3-1042-43e6-8d8b-f7c83bd26523 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:08.112737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 126/255] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 06:59:07 2022] PUT /identity/v3/projects/e594e988024e47b4a48b43d8f828c605/users/344282022c0845cba868724d0ce84528/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 06:59:08.217968 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec496a5b-380f-41fc-9c77-a95232a7d83f None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:08.243445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:08.265465 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb67710-46be-49d9-a691-2b92904c182f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:08.266529 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb67710-46be-49d9-a691-2b92904c182f None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:08.267010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb67710-46be-49d9-a691-2b92904c182f None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:08.267674 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb67710-46be-49d9-a691-2b92904c182f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:08.279082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb67710-46be-49d9-a691-2b92904c182f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:08.284833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-bfb67710-46be-49d9-a691-2b92904c182f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:59:08.284833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:08.284833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:08.284833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:08.284833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:08.284833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:08.284833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:08.284833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:08.284833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:08.284833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:08.284833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:08.284833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:08.284833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:08.284833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:08.284833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:08.284833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:08.285770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:08.285770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:08.285770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:08.285770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:08.285770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:08.285770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:08.285770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:08.285770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:08.285770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:08.285770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:08.285770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:08.285770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:08.285770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:08.285770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:08.285770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:08.285770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:08.286570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:08.286570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:08.286570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:08.286570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:08.286570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:08.286570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:08.286570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:08.286570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:08.286570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:08.286570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:08.286570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:59:08.286570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:08.289318 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 130/256] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 06:59:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:08.296558 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e9fe3-1042-43e6-8d8b-f7c83bd26523 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:08.430188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e9fe3-1042-43e6-8d8b-f7c83bd26523 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:08.480190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fde954-36f9-47a3-bcbc-a7ee348c50b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:08.481589 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fde954-36f9-47a3-bcbc-a7ee348c50b1 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:08.481874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fde954-36f9-47a3-bcbc-a7ee348c50b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:08.482149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fde954-36f9-47a3-bcbc-a7ee348c50b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:08.482844 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fde954-36f9-47a3-bcbc-a7ee348c50b1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:08.483146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fde954-36f9-47a3-bcbc-a7ee348c50b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:08.484682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fde954-36f9-47a3-bcbc-a7ee348c50b1 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:08.492811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c91f2f-2e3f-4fc4-b0cd-f3d2e81fba0c None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:08.493369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c91f2f-2e3f-4fc4-b0cd-f3d2e81fba0c None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:08.493863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c91f2f-2e3f-4fc4-b0cd-f3d2e81fba0c None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:08.495673 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 131/257] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:08.499244 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 127/258] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:08.502090 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65eec54-5fde-49bb-894d-2cd378b07b5c None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:08.502588 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65eec54-5fde-49bb-894d-2cd378b07b5c None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:08.503060 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65eec54-5fde-49bb-894d-2cd378b07b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:08.504533 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fde954-36f9-47a3-bcbc-a7ee348c50b1 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:08.523813 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-d65eec54-5fde-49bb-894d-2cd378b07b5c None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:08.525350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d65eec54-5fde-49bb-894d-2cd378b07b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:08.571866 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:08.666875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 132/259] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:59:08.694613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fde954-36f9-47a3-bcbc-a7ee348c50b1 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:08.742941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5b161-e60f-428f-8dac-ab7fabbab990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:08.743629 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5b161-e60f-428f-8dac-ab7fabbab990 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:08.743916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5b161-e60f-428f-8dac-ab7fabbab990 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:08.744196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5b161-e60f-428f-8dac-ab7fabbab990 None admin] PATH_INFO: `/v3/users/033859049778475bb121842de4286c8a` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:08.748297 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5b161-e60f-428f-8dac-ab7fabbab990 None admin] RBAC: Authorizing `identity:get_user(user_id=033859049778475bb121842de4286c8a)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:08.749782 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5b161-e60f-428f-8dac-ab7fabbab990 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:08.751430 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 128/260] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:59:08 2022] GET /identity/v3/users/033859049778475bb121842de4286c8a => generated 283 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:08.757313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65eec54-5fde-49bb-894d-2cd378b07b5c None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:08.876260 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65eec54-5fde-49bb-894d-2cd378b07b5c None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:08.887932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:08.911601 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b59f2c-8de5-4bca-bdc6-f692504d0e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:08.912212 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b59f2c-8de5-4bca-bdc6-f692504d0e32 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:08.912873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b59f2c-8de5-4bca-bdc6-f692504d0e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:08.913148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b59f2c-8de5-4bca-bdc6-f692504d0e32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:08.981825 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b59f2c-8de5-4bca-bdc6-f692504d0e32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:08.983330 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b59f2c-8de5-4bca-bdc6-f692504d0e32 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:09.049735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5b161-e60f-428f-8dac-ab7fabbab990 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:09.085990 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 133/261] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:09.091520 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b59f2c-8de5-4bca-bdc6-f692504d0e32 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:09.103760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:09.197289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b59f2c-8de5-4bca-bdc6-f692504d0e32 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:09.232685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca15284-d536-4da6-93d2-9cc07a3a0f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:09.233349 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca15284-d536-4da6-93d2-9cc07a3a0f17 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:09.233625 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca15284-d536-4da6-93d2-9cc07a3a0f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:09.233897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca15284-d536-4da6-93d2-9cc07a3a0f17 None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:09.235100 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca15284-d536-4da6-93d2-9cc07a3a0f17 None admin] RBAC: Authorizing `identity:get_project(project_id=e594e988024e47b4a48b43d8f828c605)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:09.236661 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca15284-d536-4da6-93d2-9cc07a3a0f17 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:09.238737 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 134/262] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:09 2022] GET /identity/v3/projects/e594e988024e47b4a48b43d8f828c605 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:09.244508 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca15284-d536-4da6-93d2-9cc07a3a0f17 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:09.275873 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:09.281667 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5b161-e60f-428f-8dac-ab7fabbab990 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:09.325323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:09.361225 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cc32c-53ea-4d2c-aaf2-b76ca069df00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:09.361933 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cc32c-53ea-4d2c-aaf2-b76ca069df00 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:09.362203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cc32c-53ea-4d2c-aaf2-b76ca069df00 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:09.362471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cc32c-53ea-4d2c-aaf2-b76ca069df00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:09.365525 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca15284-d536-4da6-93d2-9cc07a3a0f17 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:09.388777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cc32c-53ea-4d2c-aaf2-b76ca069df00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:09.393608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-c97cc32c-53ea-4d2c-aaf2-b76ca069df00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:59:09.393608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:09.393608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:09.393608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:09.393608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:09.393608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:09.393608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:09.393608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:09.393608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:09.393608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:09.393608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:09.393608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:09.393608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:09.393608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:09.393608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:09.393608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:09.394510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:09.394510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:09.394510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:09.394510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:09.394510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:09.394510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:09.394510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:09.394510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:09.394510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:09.394510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:09.394510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:09.394510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:09.394510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:09.394510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:09.394510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:09.394510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:09.395751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:09.395751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:09.395751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:09.395751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:09.395751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:09.395751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:09.395751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:09.395751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:09.395751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:09.395751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:09.395751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:59:09.395751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:09.395751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 129/263] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 06:59:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:09.402126 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cc32c-53ea-4d2c-aaf2-b76ca069df00 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:09.427281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c42595-ad93-494f-baf4-8dd9c4526641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:09.428048 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c42595-ad93-494f-baf4-8dd9c4526641 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:09.428412 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c42595-ad93-494f-baf4-8dd9c4526641 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:09.428746 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c42595-ad93-494f-baf4-8dd9c4526641 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:09.429582 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c42595-ad93-494f-baf4-8dd9c4526641 None admin] RBAC: Adding query filter params (role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7, scope.project.id=e594e988024e47b4a48b43d8f828c605, user.id=033859049778475bb121842de4286c8a) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:09.430011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c42595-ad93-494f-baf4-8dd9c4526641 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:09.431483 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c42595-ad93-494f-baf4-8dd9c4526641 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:09.437924 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 135/264] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:59:09 2022] GET /identity/v3/role_assignments?user.id=033859049778475bb121842de4286c8a&scope.project.id=e594e988024e47b4a48b43d8f828c605&role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:09.524029 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cc32c-53ea-4d2c-aaf2-b76ca069df00 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:09.563275 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188e588-237c-4c56-bfda-564b7f126b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:09.564208 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188e588-237c-4c56-bfda-564b7f126b7c None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:09.564627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188e588-237c-4c56-bfda-564b7f126b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:09.565036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188e588-237c-4c56-bfda-564b7f126b7c None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:09.565862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188e588-237c-4c56-bfda-564b7f126b7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:09.566320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188e588-237c-4c56-bfda-564b7f126b7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:09.567911 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188e588-237c-4c56-bfda-564b7f126b7c None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:09.579136 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 130/265] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:09.587242 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c42595-ad93-494f-baf4-8dd9c4526641 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:09.709816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c42595-ad93-494f-baf4-8dd9c4526641 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:09.745257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e7e74-8898-4329-bdf8-4ca769e05fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:09.745911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e7e74-8898-4329-bdf8-4ca769e05fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:09.746187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e7e74-8898-4329-bdf8-4ca769e05fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:09.746491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e7e74-8898-4329-bdf8-4ca769e05fb1 None admin] PATH_INFO: `/v3/users/033859049778475bb121842de4286c8a` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:09.747678 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e7e74-8898-4329-bdf8-4ca769e05fb1 None admin] RBAC: Authorizing `identity:get_user(user_id=033859049778475bb121842de4286c8a)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:09.749150 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e7e74-8898-4329-bdf8-4ca769e05fb1 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:09.751080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 136/266] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:59:09 2022] GET /identity/v3/users/033859049778475bb121842de4286c8a => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:09.756499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188e588-237c-4c56-bfda-564b7f126b7c None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:09.808976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:09.908300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188e588-237c-4c56-bfda-564b7f126b7c None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:09.947676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f572e1-9404-491f-bd16-77bdafa9bdb1 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:09.947998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f572e1-9404-491f-bd16-77bdafa9bdb1 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:09.948207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f572e1-9404-491f-bd16-77bdafa9bdb1 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:09.950589 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 137/267] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:09.956171 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fc82d-4608-47f5-8b23-1b0ec39032c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:09.956435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2428f-c42c-4ebf-be02-571ce40fde83 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:09.956648 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2428f-c42c-4ebf-be02-571ce40fde83 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:09.956921 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2428f-c42c-4ebf-be02-571ce40fde83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:09.957630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fc82d-4608-47f5-8b23-1b0ec39032c0 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:09.958101 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fc82d-4608-47f5-8b23-1b0ec39032c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:09.958573 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fc82d-4608-47f5-8b23-1b0ec39032c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:09.991080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-01b2428f-c42c-4ebf-be02-571ce40fde83 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:09.993009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-01b2428f-c42c-4ebf-be02-571ce40fde83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:10.002555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:10.066822 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530fc82d-4608-47f5-8b23-1b0ec39032c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:10.069114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530fc82d-4608-47f5-8b23-1b0ec39032c0 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:10.107732 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:10.179451 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 138/268] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:59:10.194668 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 131/269] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:10.200021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2428f-c42c-4ebf-be02-571ce40fde83 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:10.303022 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2428f-c42c-4ebf-be02-571ce40fde83 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:10.349656 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de13735-fba9-45ce-a970-38c0d5021459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:10.350224 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de13735-fba9-45ce-a970-38c0d5021459 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:10.350503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de13735-fba9-45ce-a970-38c0d5021459 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:10.350777 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de13735-fba9-45ce-a970-38c0d5021459 None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:10.356217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de13735-fba9-45ce-a970-38c0d5021459 None admin] RBAC: Authorizing `identity:get_project(project_id=e594e988024e47b4a48b43d8f828c605)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:10.357711 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de13735-fba9-45ce-a970-38c0d5021459 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:10.359697 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 139/270] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:10 2022] GET /identity/v3/projects/e594e988024e47b4a48b43d8f828c605 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:10.365174 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fc82d-4608-47f5-8b23-1b0ec39032c0 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:10.539969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ce71a-0384-4193-8c0c-11673e320ff8 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:10.539969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ce71a-0384-4193-8c0c-11673e320ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:10.539969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ce71a-0384-4193-8c0c-11673e320ff8 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:10.540666 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fc82d-4608-47f5-8b23-1b0ec39032c0 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:10.541073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 140/271] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:59:10.546619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229e48f-40c0-4194-9fe4-18ce5796a760 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:10.546901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229e48f-40c0-4194-9fe4-18ce5796a760 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:10.547181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229e48f-40c0-4194-9fe4-18ce5796a760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:10.566075 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-3229e48f-40c0-4194-9fe4-18ce5796a760 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:10.567148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3229e48f-40c0-4194-9fe4-18ce5796a760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:10.569703 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:10.584501 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:10.592736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1a831-bf4f-42de-847c-84c8bf632781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:10.593625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1a831-bf4f-42de-847c-84c8bf632781 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:10.594081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1a831-bf4f-42de-847c-84c8bf632781 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:10.594522 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1a831-bf4f-42de-847c-84c8bf632781 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:10.595600 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1a831-bf4f-42de-847c-84c8bf632781 None admin] RBAC: Adding query filter params (role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.project.id=e594e988024e47b4a48b43d8f828c605, user.id=033859049778475bb121842de4286c8a) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:10.596082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1a831-bf4f-42de-847c-84c8bf632781 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:10.597779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1a831-bf4f-42de-847c-84c8bf632781 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:10.604206 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 132/272] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:59:10 2022] GET /identity/v3/role_assignments?user.id=033859049778475bb121842de4286c8a&scope.project.id=e594e988024e47b4a48b43d8f828c605&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:10.620016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1a831-bf4f-42de-847c-84c8bf632781 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:10.666232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 141/273] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:59:10.667570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f436194-c0c4-45a9-96e4-2351e644f21d None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:10.667855 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f436194-c0c4-45a9-96e4-2351e644f21d None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:10.668125 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f436194-c0c4-45a9-96e4-2351e644f21d None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:10.669669 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 142/274] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:10.676868 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbc6f2-db56-47ea-aa74-68d651fd6255 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:10.677145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbc6f2-db56-47ea-aa74-68d651fd6255 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:10.677420 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbc6f2-db56-47ea-aa74-68d651fd6255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:10.728655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-efbbc6f2-db56-47ea-aa74-68d651fd6255 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:10.729869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-efbbc6f2-db56-47ea-aa74-68d651fd6255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:10.768322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1a831-bf4f-42de-847c-84c8bf632781 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:10.813667 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8475e-8a0e-4219-8dcd-bdf9f0a64a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:10.814316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8475e-8a0e-4219-8dcd-bdf9f0a64a9a None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:10.814598 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8475e-8a0e-4219-8dcd-bdf9f0a64a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:10.814873 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8475e-8a0e-4219-8dcd-bdf9f0a64a9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:10.827930 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8475e-8a0e-4219-8dcd-bdf9f0a64a9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:10.833671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-80f8475e-8a0e-4219-8dcd-bdf9f0a64a9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:10.833671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:10.833671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:10.833671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:10.833671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:10.833671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:10.833671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:10.833671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:10.833671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:10.833671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:10.833671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:10.833671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:10.833671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:10.833671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:10.833671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:10.833671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:10.834543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:10.834543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:10.834543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:10.834543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:10.834543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:10.834543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:10.834543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:10.834543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:10.834543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:10.834543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:10.834543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:10.834543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:10.834543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:10.834543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:10.834543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:10.834543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:10.835628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:10.835628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:10.835628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:10.835628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:10.835628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:10.835628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:10.835628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:10.835628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:10.835628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:10.835628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:10.835628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:10.835628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:10.835628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 133/275] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:10.840530 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8475e-8a0e-4219-8dcd-bdf9f0a64a9a None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:10.855328 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 143/276] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:10.924032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:10.957067 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8475e-8a0e-4219-8dcd-bdf9f0a64a9a None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:10.977923 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbc6f2-db56-47ea-aa74-68d651fd6255 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:10.995375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68daca00-ed59-40f8-843d-80bb6d49df4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:10.996493 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68daca00-ed59-40f8-843d-80bb6d49df4d None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:10.996964 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68daca00-ed59-40f8-843d-80bb6d49df4d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:10.997633 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68daca00-ed59-40f8-843d-80bb6d49df4d None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:10.998822 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68daca00-ed59-40f8-843d-80bb6d49df4d None admin] RBAC: Adding query filter params (name=member) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:10.999265 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68daca00-ed59-40f8-843d-80bb6d49df4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:11.000862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68daca00-ed59-40f8-843d-80bb6d49df4d None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:11.014814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 134/277] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:59:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:11.020320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68daca00-ed59-40f8-843d-80bb6d49df4d None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:11.070030 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:11.111935 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbc6f2-db56-47ea-aa74-68d651fd6255 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:11.145826 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68daca00-ed59-40f8-843d-80bb6d49df4d None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:11.150570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b372a-cc4b-4649-879e-ec48eafc18b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:11.151202 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b372a-cc4b-4649-879e-ec48eafc18b5 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:11.151512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b372a-cc4b-4649-879e-ec48eafc18b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:11.151783 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b372a-cc4b-4649-879e-ec48eafc18b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:11.152920 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b372a-cc4b-4649-879e-ec48eafc18b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:11.154576 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b372a-cc4b-4649-879e-ec48eafc18b5 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:11.155952 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:11.156188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 144/278] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:11.157344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75fd4f-6629-40df-96fb-2ceb1e11e7bd None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:11.157620 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75fd4f-6629-40df-96fb-2ceb1e11e7bd None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:11.157891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75fd4f-6629-40df-96fb-2ceb1e11e7bd None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:11.159069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 145/279] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:11.159775 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75fd4f-6629-40df-96fb-2ceb1e11e7bd None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:11.184593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56136e-8159-4f9a-b9a5-b129b521687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:11.185307 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56136e-8159-4f9a-b9a5-b129b521687f None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:11.185579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56136e-8159-4f9a-b9a5-b129b521687f None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:11.185848 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56136e-8159-4f9a-b9a5-b129b521687f None admin] PATH_INFO: `/v3/users/d157873e84564a86a03bc9899fbf6711` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:11.199476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56136e-8159-4f9a-b9a5-b129b521687f None admin] RBAC: Authorizing `identity:get_user(user_id=d157873e84564a86a03bc9899fbf6711)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:11.201480 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56136e-8159-4f9a-b9a5-b129b521687f None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:11.203253 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 135/280] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:59:11 2022] GET /identity/v3/users/d157873e84564a86a03bc9899fbf6711 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:11.203908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56136e-8159-4f9a-b9a5-b129b521687f None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:11.262733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75fd4f-6629-40df-96fb-2ceb1e11e7bd None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:11.295393 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba69f8-d903-4569-a169-69c4ac46975b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:11.296042 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba69f8-d903-4569-a169-69c4ac46975b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:11.296323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba69f8-d903-4569-a169-69c4ac46975b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:11.296593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba69f8-d903-4569-a169-69c4ac46975b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:11.298853 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56136e-8159-4f9a-b9a5-b129b521687f None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:11.299143 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:11.307660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba69f8-d903-4569-a169-69c4ac46975b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:11.312686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-bdba69f8-d903-4569-a169-69c4ac46975b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:11.312686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:11.312686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:11.312686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:11.312686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:11.312686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:11.312686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:11.312686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:11.312686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:11.312686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:11.312686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:11.312686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:11.312686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:11.312686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:11.312686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:11.312686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:11.313581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:11.313581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:11.313581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:11.313581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:11.313581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:11.313581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:11.313581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:11.313581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:11.313581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:11.313581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:11.313581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:11.313581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:11.313581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:11.313581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:11.313581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:11.313581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:11.314593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:11.314593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:11.314593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:11.314593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:11.314593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:11.314593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:11.314593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:11.314593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:11.314593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:11.314593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:11.314593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:11.314593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:11.314593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 146/281] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:11.320020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4a7a6-1605-4171-b8bb-47c883438dcc None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:11.320312 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4a7a6-1605-4171-b8bb-47c883438dcc None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:11.320586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4a7a6-1605-4171-b8bb-47c883438dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:11.334114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d293ed0-5df6-4c40-8efa-622fa2404e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:11.334720 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d293ed0-5df6-4c40-8efa-622fa2404e13 None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:11.334994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d293ed0-5df6-4c40-8efa-622fa2404e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:11.335258 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d293ed0-5df6-4c40-8efa-622fa2404e13 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:11.335925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d293ed0-5df6-4c40-8efa-622fa2404e13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:11.337367 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d293ed0-5df6-4c40-8efa-622fa2404e13 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:11.338044 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-8ee4a7a6-1605-4171-b8bb-47c883438dcc None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:11.339235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8ee4a7a6-1605-4171-b8bb-47c883438dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:11.357450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 136/282] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 23 06:59:11 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:59:11.357450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d293ed0-5df6-4c40-8efa-622fa2404e13 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:11.388081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:11.429806 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 147/283] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:11.439672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4a7a6-1605-4171-b8bb-47c883438dcc None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:11.497837 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d293ed0-5df6-4c40-8efa-622fa2404e13 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:11.544403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25704a7-b3a8-4446-9833-b785b781c9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:11.545358 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25704a7-b3a8-4446-9833-b785b781c9f6 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:11.545642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25704a7-b3a8-4446-9833-b785b781c9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:11.546192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25704a7-b3a8-4446-9833-b785b781c9f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:11.546943 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25704a7-b3a8-4446-9833-b785b781c9f6 None admin] RBAC: Adding query filter params (role.id=7670c7c450b9404ba57e12743cb6c458, user.id=d157873e84564a86a03bc9899fbf6711, scope.system=all) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:11.547434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25704a7-b3a8-4446-9833-b785b781c9f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:11.549420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25704a7-b3a8-4446-9833-b785b781c9f6 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:11.553697 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4a7a6-1605-4171-b8bb-47c883438dcc None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:11.554660 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 137/284] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Aug 23 06:59:11 2022] GET /identity/v3/role_assignments?user.id=d157873e84564a86a03bc9899fbf6711&scope.system=all&role.id=7670c7c450b9404ba57e12743cb6c458 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:11.593075 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:11.604493 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac36d3-edb9-47b6-9d94-e6da12d440d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:11.605568 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac36d3-edb9-47b6-9d94-e6da12d440d0 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:11.606051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac36d3-edb9-47b6-9d94-e6da12d440d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:11.606673 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac36d3-edb9-47b6-9d94-e6da12d440d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:11.607865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac36d3-edb9-47b6-9d94-e6da12d440d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:11.608409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac36d3-edb9-47b6-9d94-e6da12d440d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:11.610073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac36d3-edb9-47b6-9d94-e6da12d440d0 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:11.621081 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 148/285] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:59:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:11.627352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25704a7-b3a8-4446-9833-b785b781c9f6 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:11.724769 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac36d3-edb9-47b6-9d94-e6da12d440d0 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:11.744622 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:11.767471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25704a7-b3a8-4446-9833-b785b781c9f6 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:11.812045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68584cf8-502c-41a7-90a8-2306c72edaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:11.812695 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68584cf8-502c-41a7-90a8-2306c72edaaa None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:11.812979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68584cf8-502c-41a7-90a8-2306c72edaaa None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:11.813255 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68584cf8-502c-41a7-90a8-2306c72edaaa None admin] PATH_INFO: `/v3/users/344282022c0845cba868724d0ce84528` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:11.814602 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68584cf8-502c-41a7-90a8-2306c72edaaa None admin] RBAC: Authorizing `identity:get_user(user_id=344282022c0845cba868724d0ce84528)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:11.816172 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68584cf8-502c-41a7-90a8-2306c72edaaa None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:11.818205 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 138/286] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:59:11 2022] GET /identity/v3/users/344282022c0845cba868724d0ce84528 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:11.825281 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68584cf8-502c-41a7-90a8-2306c72edaaa None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:11.874537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac36d3-edb9-47b6-9d94-e6da12d440d0 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:11.918313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc26fa-9be7-46bd-ac1f-b9d984b77ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:11.919127 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc26fa-9be7-46bd-ac1f-b9d984b77ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:11.919404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc26fa-9be7-46bd-ac1f-b9d984b77ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:11.920163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc26fa-9be7-46bd-ac1f-b9d984b77ae0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:11.933651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc26fa-9be7-46bd-ac1f-b9d984b77ae0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:11.936044 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68584cf8-502c-41a7-90a8-2306c72edaaa None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:11.939865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-f2cc26fa-9be7-46bd-ac1f-b9d984b77ae0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 06:59:11.939865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:11.939865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:11.939865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:11.939865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:11.939865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:11.939865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:11.939865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:11.939865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:11.939865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:11.939865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:11.939865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:11.939865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:11.939865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:11.939865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:11.939865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:11.940734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:11.940734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:11.940734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:11.940734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:11.940734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:11.940734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:11.940734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:11.940734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:11.940734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:11.940734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:11.940734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:11.940734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:11.940734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:11.940734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:11.940734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:11.940734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:11.941528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:11.941528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:11.941528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:11.941528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:11.941528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:11.941528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:11.941528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:11.941528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:11.941528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:11.941528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:11.941528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 06:59:11.941528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:11.941528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 149/287] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:11.946966 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc26fa-9be7-46bd-ac1f-b9d984b77ae0 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:11.991536 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424adc0-2683-43fe-981a-eaf0387c86f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:11.992156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424adc0-2683-43fe-981a-eaf0387c86f7 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:11.992437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424adc0-2683-43fe-981a-eaf0387c86f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:11.992710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424adc0-2683-43fe-981a-eaf0387c86f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:12.043818 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:12.051459 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:12.117433 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6424adc0-2683-43fe-981a-eaf0387c86f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:12.119101 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6424adc0-2683-43fe-981a-eaf0387c86f7 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:12.139147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc26fa-9be7-46bd-ac1f-b9d984b77ae0 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:12.177417 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab59c3-512e-4af9-9c9e-abed09e70d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:12.178409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab59c3-512e-4af9-9c9e-abed09e70d4d None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:12.178876 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab59c3-512e-4af9-9c9e-abed09e70d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:12.179333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab59c3-512e-4af9-9c9e-abed09e70d4d None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:12.180228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab59c3-512e-4af9-9c9e-abed09e70d4d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:12.180714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab59c3-512e-4af9-9c9e-abed09e70d4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:12.182298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab59c3-512e-4af9-9c9e-abed09e70d4d None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:12.191748 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 150/288] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:59:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:12.204486 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab59c3-512e-4af9-9c9e-abed09e70d4d None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:12.243232 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 139/289] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:12.249144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424adc0-2683-43fe-981a-eaf0387c86f7 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:12.313013 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:12.363465 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424adc0-2683-43fe-981a-eaf0387c86f7 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:12.402318 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad9048-fe4b-4313-a63f-3116b6b382c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:12.402967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad9048-fe4b-4313-a63f-3116b6b382c5 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:12.403304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad9048-fe4b-4313-a63f-3116b6b382c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:12.403613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad9048-fe4b-4313-a63f-3116b6b382c5 None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:12.404754 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad9048-fe4b-4313-a63f-3116b6b382c5 None admin] RBAC: Authorizing `identity:get_project(project_id=e594e988024e47b4a48b43d8f828c605)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:12.406226 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad9048-fe4b-4313-a63f-3116b6b382c5 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:12.408514 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 140/290] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:12 2022] GET /identity/v3/projects/e594e988024e47b4a48b43d8f828c605 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:12.415079 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad9048-fe4b-4313-a63f-3116b6b382c5 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:12.465439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab59c3-512e-4af9-9c9e-abed09e70d4d None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:12.491778 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:12.533012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad9048-fe4b-4313-a63f-3116b6b382c5 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:12.559097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:12.568647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948a59c-6209-458f-82d2-7ec4b2ed8181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:12.569617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948a59c-6209-458f-82d2-7ec4b2ed8181 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:12.570000 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948a59c-6209-458f-82d2-7ec4b2ed8181 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:12.570275 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948a59c-6209-458f-82d2-7ec4b2ed8181 None admin] PATH_INFO: `/v3/users/a1c58bd4231345c28b05c6987217d213` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:12.571433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0948a59c-6209-458f-82d2-7ec4b2ed8181 None admin] RBAC: Authorizing `identity:get_user(user_id=a1c58bd4231345c28b05c6987217d213)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:12.573025 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464ae49-6420-4283-8654-ca8a81eaeaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:12.573146 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0948a59c-6209-458f-82d2-7ec4b2ed8181 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:12.573642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464ae49-6420-4283-8654-ca8a81eaeaf7 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:12.573920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464ae49-6420-4283-8654-ca8a81eaeaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:12.574193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464ae49-6420-4283-8654-ca8a81eaeaf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:12.574922 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464ae49-6420-4283-8654-ca8a81eaeaf7 None admin] RBAC: Adding query filter params (role.id=7670c7c450b9404ba57e12743cb6c458, scope.project.id=e594e988024e47b4a48b43d8f828c605, user.id=344282022c0845cba868724d0ce84528) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:12.575359 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 151/291] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:59:12 2022] GET /identity/v3/users/a1c58bd4231345c28b05c6987217d213 => generated 297 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:12.575792 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464ae49-6420-4283-8654-ca8a81eaeaf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:12.581723 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464ae49-6420-4283-8654-ca8a81eaeaf7 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:12.584322 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948a59c-6209-458f-82d2-7ec4b2ed8181 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:12.588503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 141/292] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:59:12 2022] GET /identity/v3/role_assignments?user.id=344282022c0845cba868724d0ce84528&scope.project.id=e594e988024e47b4a48b43d8f828c605&role.id=7670c7c450b9404ba57e12743cb6c458 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:12.784444 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948a59c-6209-458f-82d2-7ec4b2ed8181 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:12.827964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b07dad-8979-4714-9ed5-575fb7b87c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:12.828940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b07dad-8979-4714-9ed5-575fb7b87c2d None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:12.829431 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b07dad-8979-4714-9ed5-575fb7b87c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:12.829891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b07dad-8979-4714-9ed5-575fb7b87c2d None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:12.831199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b07dad-8979-4714-9ed5-575fb7b87c2d None admin] RBAC: Authorizing `identity:get_project(project_id=e594e988024e47b4a48b43d8f828c605)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:12.832859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b07dad-8979-4714-9ed5-575fb7b87c2d None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:12.834973 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 152/293] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:12 2022] GET /identity/v3/projects/e594e988024e47b4a48b43d8f828c605 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:12.840984 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464ae49-6420-4283-8654-ca8a81eaeaf7 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:12.956736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464ae49-6420-4283-8654-ca8a81eaeaf7 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:12.995974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f96d9-473f-4f0a-b021-598c9dd34a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:12.996707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f96d9-473f-4f0a-b021-598c9dd34a0e None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:12.997055 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f96d9-473f-4f0a-b021-598c9dd34a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:12.997329 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f96d9-473f-4f0a-b021-598c9dd34a0e None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605/users/a1c58bd4231345c28b05c6987217d213/roles/ff4e7aa3dd7e4272a3cd1cafb32baa24` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:12.999270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f96d9-473f-4f0a-b021-598c9dd34a0e None admin] RBAC: Authorizing `identity:create_grant(project_id=e594e988024e47b4a48b43d8f828c605, user_id=a1c58bd4231345c28b05c6987217d213, role_id=ff4e7aa3dd7e4272a3cd1cafb32baa24)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:13.000770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f96d9-473f-4f0a-b021-598c9dd34a0e None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:13.036658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 142/294] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 06:59:12 2022] PUT /identity/v3/projects/e594e988024e47b4a48b43d8f828c605/users/a1c58bd4231345c28b05c6987217d213/roles/ff4e7aa3dd7e4272a3cd1cafb32baa24 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 06:59:13.585244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3afe4-e6ef-492c-9398-2028b730f0f5 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:13.585793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3afe4-e6ef-492c-9398-2028b730f0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:13.585793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3afe4-e6ef-492c-9398-2028b730f0f5 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:13.587430 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 153/295] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:13.592872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2019c-4d98-45a2-bd5b-3d7d5fbdb9cf None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:13.593158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2019c-4d98-45a2-bd5b-3d7d5fbdb9cf None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:13.593437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2019c-4d98-45a2-bd5b-3d7d5fbdb9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:13.612927 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-c1f2019c-4d98-45a2-bd5b-3d7d5fbdb9cf None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:13.614204 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c1f2019c-4d98-45a2-bd5b-3d7d5fbdb9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:13.660303 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:13.691546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 143/296] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:13.732393 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1303b-82c2-4674-ab2f-264d861c1a7e None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:13.732720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1303b-82c2-4674-ab2f-264d861c1a7e None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:13.732942 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1303b-82c2-4674-ab2f-264d861c1a7e None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:13.734567 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 154/297] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:13.739944 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bc91e-699b-4339-9a3c-51d1b4f61899 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:13.740396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bc91e-699b-4339-9a3c-51d1b4f61899 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:13.740803 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bc91e-699b-4339-9a3c-51d1b4f61899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:13.759091 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-1a2bc91e-699b-4339-9a3c-51d1b4f61899 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:13.760496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-1a2bc91e-699b-4339-9a3c-51d1b4f61899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:13.828936 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 144/298] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:59:13.935717 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70f1a1-f33f-438d-b1ce-203532d07677 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:13.936003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70f1a1-f33f-438d-b1ce-203532d07677 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:13.936278 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70f1a1-f33f-438d-b1ce-203532d07677 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:13.937886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 155/299] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:13.943080 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4246c8-3513-4daf-afb8-0050a40f39d5 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:13.943584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4246c8-3513-4daf-afb8-0050a40f39d5 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:13.943662 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4246c8-3513-4daf-afb8-0050a40f39d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:13.965705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-cf4246c8-3513-4daf-afb8-0050a40f39d5 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:13.967111 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-cf4246c8-3513-4daf-afb8-0050a40f39d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:13.975294 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70f1a1-f33f-438d-b1ce-203532d07677 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:14.036168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 145/300] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:59:14.039793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:14.092062 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70f1a1-f33f-438d-b1ce-203532d07677 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:14.104475 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4246c8-3513-4daf-afb8-0050a40f39d5 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:14.134494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bedbdc-46aa-401e-8274-4ab438aeef19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:14.136126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bedbdc-46aa-401e-8274-4ab438aeef19 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:14.136626 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bedbdc-46aa-401e-8274-4ab438aeef19 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:14.137323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bedbdc-46aa-401e-8274-4ab438aeef19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:14.142927 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:14.153767 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bedbdc-46aa-401e-8274-4ab438aeef19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:14.161352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-93bedbdc-46aa-401e-8274-4ab438aeef19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:14.161352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:14.161352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:14.161352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:14.161352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:14.161352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:14.161352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:14.161352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:14.161352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:14.161352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:14.161352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:14.161352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:14.161352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:14.161352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:14.161352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:14.161352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:14.162221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:14.162221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:14.162221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:14.162221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:14.162221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:14.162221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:14.162221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:14.162221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:14.162221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:14.162221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:14.162221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:14.162221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:14.162221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:14.162221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:14.162221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:14.162221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:14.162995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:14.162995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:14.162995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:14.162995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:14.162995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:14.162995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:14.162995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:14.162995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:14.162995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:14.162995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:14.162995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:14.162995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:14.164796 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 156/301] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:14.171280 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bedbdc-46aa-401e-8274-4ab438aeef19 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:14.217409 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4246c8-3513-4daf-afb8-0050a40f39d5 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:14.263345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6f261-aa14-4e51-a0a6-0884f2143e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:14.264441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6f261-aa14-4e51-a0a6-0884f2143e88 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:14.264930 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6f261-aa14-4e51-a0a6-0884f2143e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:14.265865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6f261-aa14-4e51-a0a6-0884f2143e88 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:14.279632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6f261-aa14-4e51-a0a6-0884f2143e88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:14.284628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-63e6f261-aa14-4e51-a0a6-0884f2143e88 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 06:59:14.284628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:14.284628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:14.284628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:14.284628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:14.284628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:14.284628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:14.284628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:14.284628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:14.284628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:14.284628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:14.284628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:14.284628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:14.284628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:14.284628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:14.284628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:14.285503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:14.285503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:14.285503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:14.285503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:14.285503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:14.285503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:14.285503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:14.285503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:14.285503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:14.285503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:14.285503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:14.285503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:14.285503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:14.285503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:14.285503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:14.285503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:14.286327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:14.286327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:14.286327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:14.286327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:14.286327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:14.286327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:14.286327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:14.286327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:14.286327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:14.286327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:14.286327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 06:59:14.286327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:14.290779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 146/302] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:14.291763 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6f261-aa14-4e51-a0a6-0884f2143e88 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:14.298752 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bedbdc-46aa-401e-8274-4ab438aeef19 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:14.307277 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:14.338813 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c86b2a-979e-4a1c-9d01-53099f9f94b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:14.339446 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c86b2a-979e-4a1c-9d01-53099f9f94b5 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:14.339828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c86b2a-979e-4a1c-9d01-53099f9f94b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:14.340096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c86b2a-979e-4a1c-9d01-53099f9f94b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:14.340755 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c86b2a-979e-4a1c-9d01-53099f9f94b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:14.341080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c86b2a-979e-4a1c-9d01-53099f9f94b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:14.342530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c86b2a-979e-4a1c-9d01-53099f9f94b5 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:14.350895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 157/303] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:59:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:14.352368 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c86b2a-979e-4a1c-9d01-53099f9f94b5 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:14.389816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:14.396102 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6f261-aa14-4e51-a0a6-0884f2143e88 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:14.435006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e0be0-36e5-41e9-9d83-78d44fe0bd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:14.435817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e0be0-36e5-41e9-9d83-78d44fe0bd56 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:14.436098 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e0be0-36e5-41e9-9d83-78d44fe0bd56 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:14.436373 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e0be0-36e5-41e9-9d83-78d44fe0bd56 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:14.437042 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e0be0-36e5-41e9-9d83-78d44fe0bd56 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:14.437340 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e0be0-36e5-41e9-9d83-78d44fe0bd56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:14.438729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e0be0-36e5-41e9-9d83-78d44fe0bd56 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:14.452752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 147/304] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:59:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:14.452752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e0be0-36e5-41e9-9d83-78d44fe0bd56 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:14.463009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c86b2a-979e-4a1c-9d01-53099f9f94b5 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:14.497024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4733a5-ad64-45f2-a7de-adbab106104c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:14.497640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4733a5-ad64-45f2-a7de-adbab106104c None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:14.498113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4733a5-ad64-45f2-a7de-adbab106104c None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:14.498469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4733a5-ad64-45f2-a7de-adbab106104c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:14.499605 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4733a5-ad64-45f2-a7de-adbab106104c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:14.504985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4733a5-ad64-45f2-a7de-adbab106104c None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:14.504985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 158/305] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:14.504985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4733a5-ad64-45f2-a7de-adbab106104c None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:14.546638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:14.570552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e0be0-36e5-41e9-9d83-78d44fe0bd56 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:14.608812 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a901280-32d9-4416-b676-1ccd9b31c5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:14.609444 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a901280-32d9-4416-b676-1ccd9b31c5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:14.609721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a901280-32d9-4416-b676-1ccd9b31c5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:14.610032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a901280-32d9-4416-b676-1ccd9b31c5e3 None admin] PATH_INFO: `/v3/users/d157873e84564a86a03bc9899fbf6711` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:14.611107 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a901280-32d9-4416-b676-1ccd9b31c5e3 None admin] RBAC: Authorizing `identity:get_user(user_id=d157873e84564a86a03bc9899fbf6711)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:14.612603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a901280-32d9-4416-b676-1ccd9b31c5e3 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:14.614166 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 148/306] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:59:14 2022] GET /identity/v3/users/d157873e84564a86a03bc9899fbf6711 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:14.614905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a901280-32d9-4416-b676-1ccd9b31c5e3 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:14.624119 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4733a5-ad64-45f2-a7de-adbab106104c None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:14.646221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:14.670667 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dfb91-d6b3-4880-a073-93def32b5497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:14.671395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dfb91-d6b3-4880-a073-93def32b5497 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:14.671791 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dfb91-d6b3-4880-a073-93def32b5497 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:14.672139 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dfb91-d6b3-4880-a073-93def32b5497 None admin] PATH_INFO: `/v3/users/7b05b363aae4464ead3a8674e2d53015` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:14.691582 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954dfb91-d6b3-4880-a073-93def32b5497 None admin] RBAC: Authorizing `identity:get_user(user_id=7b05b363aae4464ead3a8674e2d53015)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:14.693148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954dfb91-d6b3-4880-a073-93def32b5497 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:14.695318 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 159/307] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:59:14 2022] GET /identity/v3/users/7b05b363aae4464ead3a8674e2d53015 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:14.700763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dfb91-d6b3-4880-a073-93def32b5497 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:14.731327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a901280-32d9-4416-b676-1ccd9b31c5e3 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:14.768220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7494d7d-41d3-428d-892c-94ce8971ee3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:14.768758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7494d7d-41d3-428d-892c-94ce8971ee3e None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:14.769029 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7494d7d-41d3-428d-892c-94ce8971ee3e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:14.769303 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7494d7d-41d3-428d-892c-94ce8971ee3e None admin] PATH_INFO: `/v3/groups` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:14.769960 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7494d7d-41d3-428d-892c-94ce8971ee3e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:14.771452 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7494d7d-41d3-428d-892c-94ce8971ee3e None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:14.795447 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 149/308] 10.222.0.17 () {60 vars in 1134 bytes} [Tue Aug 23 06:59:14 2022] POST /identity/v3/groups => generated 229 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:59:14.795447 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7494d7d-41d3-428d-892c-94ce8971ee3e None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:14.812713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:14.822746 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dfb91-d6b3-4880-a073-93def32b5497 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:14.872005 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e803e6-2a97-4bdb-b28b-d20cc87be446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:14.873000 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e803e6-2a97-4bdb-b28b-d20cc87be446 None admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:14.874082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e803e6-2a97-4bdb-b28b-d20cc87be446 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:14.874476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e803e6-2a97-4bdb-b28b-d20cc87be446 None admin] PATH_INFO: `/v3/system/users/d157873e84564a86a03bc9899fbf6711/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:14.877027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e803e6-2a97-4bdb-b28b-d20cc87be446 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d157873e84564a86a03bc9899fbf6711, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:14.879863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e803e6-2a97-4bdb-b28b-d20cc87be446 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:14.892709 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 160/309] 10.222.0.17 () {58 vars in 1334 bytes} [Tue Aug 23 06:59:14 2022] PUT /identity/v3/system/users/d157873e84564a86a03bc9899fbf6711/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:59:14.902048 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:14.926225 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7494d7d-41d3-428d-892c-94ce8971ee3e None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:14.969192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a95dd4-460e-4f4a-b42d-dfa1ffba121e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:14.969826 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a95dd4-460e-4f4a-b42d-dfa1ffba121e None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:14.970099 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a95dd4-460e-4f4a-b42d-dfa1ffba121e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:14.970374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a95dd4-460e-4f4a-b42d-dfa1ffba121e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:14.971235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a95dd4-460e-4f4a-b42d-dfa1ffba121e None admin] RBAC: Adding query filter params (role.id=ff4e7aa3dd7e4272a3cd1cafb32baa24, user.id=7b05b363aae4464ead3a8674e2d53015, scope.system=all) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:14.971588 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a95dd4-460e-4f4a-b42d-dfa1ffba121e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:14.973084 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a95dd4-460e-4f4a-b42d-dfa1ffba121e None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:14.980318 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 150/310] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Aug 23 06:59:14 2022] GET /identity/v3/role_assignments?user.id=7b05b363aae4464ead3a8674e2d53015&scope.system=all&role.id=ff4e7aa3dd7e4272a3cd1cafb32baa24 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:15.096865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888d374-fab5-4742-98c3-1da695e0d5da None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:15.097168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888d374-fab5-4742-98c3-1da695e0d5da None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:15.097342 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888d374-fab5-4742-98c3-1da695e0d5da None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:15.098736 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 161/311] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:15.104669 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f376034-dd8f-4b5b-91ad-0ac38268f3c9 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:15.104950 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f376034-dd8f-4b5b-91ad-0ac38268f3c9 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:15.105226 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f376034-dd8f-4b5b-91ad-0ac38268f3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:15.125789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-5f376034-dd8f-4b5b-91ad-0ac38268f3c9 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:15.127017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-5f376034-dd8f-4b5b-91ad-0ac38268f3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:15.211110 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 151/312] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:15.485119 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888d374-fab5-4742-98c3-1da695e0d5da None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:15.604346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888d374-fab5-4742-98c3-1da695e0d5da None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:15.645119 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7ab83-0377-4e08-82e4-7ead465471c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:15.645754 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7ab83-0377-4e08-82e4-7ead465471c1 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:15.646008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7ab83-0377-4e08-82e4-7ead465471c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:15.646285 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7ab83-0377-4e08-82e4-7ead465471c1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:15.661486 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7ab83-0377-4e08-82e4-7ead465471c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:15.666472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-73f7ab83-0377-4e08-82e4-7ead465471c1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 06:59:15.666472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:15.666472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:15.666472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:15.666472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:15.666472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:15.666472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:15.666472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:15.666472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:15.666472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:15.666472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:15.666472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:15.666472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:15.666472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:15.666472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:15.666472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:15.667333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:15.667333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:15.667333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:15.667333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:15.667333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:15.667333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:15.667333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:15.667333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:15.667333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:15.667333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:15.667333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:15.667333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:15.667333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:15.667333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:15.667333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:15.667333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:15.668157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:15.668157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:15.668157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:15.668157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:15.668157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:15.668157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:15.668157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:15.668157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:15.668157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:15.668157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:15.668157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 06:59:15.668157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:15.668157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 162/313] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:15.673728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f376034-dd8f-4b5b-91ad-0ac38268f3c9 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:15.759474 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:15.779696 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f376034-dd8f-4b5b-91ad-0ac38268f3c9 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:15.814946 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f11e4d-7836-4614-8ede-9c982bd84968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:15.815602 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f11e4d-7836-4614-8ede-9c982bd84968 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:15.815881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f11e4d-7836-4614-8ede-9c982bd84968 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:15.816154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f11e4d-7836-4614-8ede-9c982bd84968 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:15.816830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f11e4d-7836-4614-8ede-9c982bd84968 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:15.817129 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f11e4d-7836-4614-8ede-9c982bd84968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:15.818520 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f11e4d-7836-4614-8ede-9c982bd84968 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:15.828014 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 152/314] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:59:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:15.833570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7ab83-0377-4e08-82e4-7ead465471c1 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:15.857045 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:15.943637 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7ab83-0377-4e08-82e4-7ead465471c1 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:15.979693 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816af081-df49-4e8d-b20e-7c17e01fcbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:15.980686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816af081-df49-4e8d-b20e-7c17e01fcbb7 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:15.981182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816af081-df49-4e8d-b20e-7c17e01fcbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:15.981622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816af081-df49-4e8d-b20e-7c17e01fcbb7 None admin] PATH_INFO: `/v3/users/a1c58bd4231345c28b05c6987217d213` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:15.983581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816af081-df49-4e8d-b20e-7c17e01fcbb7 None admin] RBAC: Authorizing `identity:get_user(user_id=a1c58bd4231345c28b05c6987217d213)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:15.985243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816af081-df49-4e8d-b20e-7c17e01fcbb7 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:15.987346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 163/315] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:59:15 2022] GET /identity/v3/users/a1c58bd4231345c28b05c6987217d213 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:15.993292 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f11e4d-7836-4614-8ede-9c982bd84968 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:16.109404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f11e4d-7836-4614-8ede-9c982bd84968 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:16.144245 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fd2ba-4e78-4ba5-b67f-033ab9abaea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:16.145163 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fd2ba-4e78-4ba5-b67f-033ab9abaea9 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:16.145615 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fd2ba-4e78-4ba5-b67f-033ab9abaea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:16.146100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fd2ba-4e78-4ba5-b67f-033ab9abaea9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:16.159797 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:16.218780 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8fd2ba-4e78-4ba5-b67f-033ab9abaea9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:16.220282 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8fd2ba-4e78-4ba5-b67f-033ab9abaea9 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:16.322656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 153/316] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:16.329136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816af081-df49-4e8d-b20e-7c17e01fcbb7 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:16.419346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:16.435066 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816af081-df49-4e8d-b20e-7c17e01fcbb7 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:16.471145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df85859-30d2-4998-8ecd-9de5c36a2a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:16.471817 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df85859-30d2-4998-8ecd-9de5c36a2a13 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:16.472096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df85859-30d2-4998-8ecd-9de5c36a2a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:16.472367 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df85859-30d2-4998-8ecd-9de5c36a2a13 None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:16.473489 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df85859-30d2-4998-8ecd-9de5c36a2a13 None admin] RBAC: Authorizing `identity:get_project(project_id=e594e988024e47b4a48b43d8f828c605)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:16.474941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df85859-30d2-4998-8ecd-9de5c36a2a13 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:16.476641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 164/317] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:16 2022] GET /identity/v3/projects/e594e988024e47b4a48b43d8f828c605 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:16.483293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fd2ba-4e78-4ba5-b67f-033ab9abaea9 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:16.529362 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:16.590520 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fd2ba-4e78-4ba5-b67f-033ab9abaea9 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:16.628195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6d34c-bb3c-4117-a013-6f7bc6225987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:16.628803 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6d34c-bb3c-4117-a013-6f7bc6225987 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:16.629133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6d34c-bb3c-4117-a013-6f7bc6225987 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:16.629405 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6d34c-bb3c-4117-a013-6f7bc6225987 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:16.630140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6d34c-bb3c-4117-a013-6f7bc6225987 None admin] RBAC: Adding query filter params (role.id=ff4e7aa3dd7e4272a3cd1cafb32baa24, scope.project.id=e594e988024e47b4a48b43d8f828c605, user.id=a1c58bd4231345c28b05c6987217d213) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:16.630442 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6d34c-bb3c-4117-a013-6f7bc6225987 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:16.631924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6d34c-bb3c-4117-a013-6f7bc6225987 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:16.638164 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 154/318] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:59:16 2022] GET /identity/v3/role_assignments?user.id=a1c58bd4231345c28b05c6987217d213&scope.project.id=e594e988024e47b4a48b43d8f828c605&role.id=ff4e7aa3dd7e4272a3cd1cafb32baa24 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:16.884969 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367917c-2aae-4de9-b4e0-17f814d5bbde None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:16.885169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac34b54-b884-40d3-b48a-cc68fc4285f4 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:16.885591 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367917c-2aae-4de9-b4e0-17f814d5bbde None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:16.885698 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac34b54-b884-40d3-b48a-cc68fc4285f4 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:16.885770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367917c-2aae-4de9-b4e0-17f814d5bbde None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:16.885835 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac34b54-b884-40d3-b48a-cc68fc4285f4 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:16.886997 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 165/319] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:16.887096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 155/320] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:16.891933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236eebc7-54b2-479f-ad02-02cfc486c353 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:16.892170 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cf82d-c569-4b85-857b-610c69c2db6f None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:16.892342 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236eebc7-54b2-479f-ad02-02cfc486c353 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:16.892433 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cf82d-c569-4b85-857b-610c69c2db6f None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:16.892611 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236eebc7-54b2-479f-ad02-02cfc486c353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:16.892714 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cf82d-c569-4b85-857b-610c69c2db6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:16.911683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-236eebc7-54b2-479f-ad02-02cfc486c353 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:16.912898 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-236eebc7-54b2-479f-ad02-02cfc486c353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:16.913004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-861cf82d-c569-4b85-857b-610c69c2db6f None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:16.914156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-861cf82d-c569-4b85-857b-610c69c2db6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:16.978395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 166/321] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:16.979658 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587094e-8dfd-4384-8d7a-05f426c0c23f None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:16.979940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587094e-8dfd-4384-8d7a-05f426c0c23f None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:16.980206 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587094e-8dfd-4384-8d7a-05f426c0c23f None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:16.981439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 167/322] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:16.985911 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 156/323] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:59:16.986808 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225aaffd-6d6d-4cbb-9c78-b9db3e3a7a84 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:16.987086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225aaffd-6d6d-4cbb-9c78-b9db3e3a7a84 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:16.987380 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225aaffd-6d6d-4cbb-9c78-b9db3e3a7a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:17.005892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-225aaffd-6d6d-4cbb-9c78-b9db3e3a7a84 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:17.007145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-225aaffd-6d6d-4cbb-9c78-b9db3e3a7a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:17.074165 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 168/324] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:17.242876 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cf82d-c569-4b85-857b-610c69c2db6f None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:17.248028 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225aaffd-6d6d-4cbb-9c78-b9db3e3a7a84 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:17.263988 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:17.285016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:17.353745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cf82d-c569-4b85-857b-610c69c2db6f None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:17.356366 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225aaffd-6d6d-4cbb-9c78-b9db3e3a7a84 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:17.387205 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde05e3-1981-49a1-975b-2f1bea0af649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:17.387872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde05e3-1981-49a1-975b-2f1bea0af649 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:17.388165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde05e3-1981-49a1-975b-2f1bea0af649 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:17.388489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde05e3-1981-49a1-975b-2f1bea0af649 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:17.390467 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293c9e1-af33-4b5b-8673-b4c14a2dd9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:17.391093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293c9e1-af33-4b5b-8673-b4c14a2dd9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:17.391367 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293c9e1-af33-4b5b-8673-b4c14a2dd9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:17.391667 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293c9e1-af33-4b5b-8673-b4c14a2dd9c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:17.392759 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293c9e1-af33-4b5b-8673-b4c14a2dd9c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:17.394208 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293c9e1-af33-4b5b-8673-b4c14a2dd9c0 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:17.395851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 169/325] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:17.398864 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde05e3-1981-49a1-975b-2f1bea0af649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:17.399777 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293c9e1-af33-4b5b-8673-b4c14a2dd9c0 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:17.403953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-3fde05e3-1981-49a1-975b-2f1bea0af649 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:17.403953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:17.403953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:17.403953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:17.403953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:17.403953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:17.403953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:17.403953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:17.403953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:17.403953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:17.403953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:17.403953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:17.403953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:17.403953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:17.403953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:17.403953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:17.405502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:17.405502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:17.405502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:17.405502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:17.405502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:17.405502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:17.405502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:17.405502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:17.405502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:17.405502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:17.405502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:17.405502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:17.405502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:17.405502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:17.405502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:17.405502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:17.406497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:17.406497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:17.406497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:17.406497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:17.406497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:17.406497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:17.406497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:17.406497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:17.406497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:17.406497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:17.406497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:17.406497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:17.406497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 157/326] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:17.408682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde05e3-1981-49a1-975b-2f1bea0af649 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:17.480610 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:17.499884 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293c9e1-af33-4b5b-8673-b4c14a2dd9c0 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:17.501827 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:17.507865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde05e3-1981-49a1-975b-2f1bea0af649 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:17.534351 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e147b29-7684-4d53-87f6-b286184436d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:17.534978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e147b29-7684-4d53-87f6-b286184436d7 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:17.535252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e147b29-7684-4d53-87f6-b286184436d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:17.535548 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e147b29-7684-4d53-87f6-b286184436d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:17.541920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa912c1-128b-4217-ba9e-9256c0fbc656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:17.542544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa912c1-128b-4217-ba9e-9256c0fbc656 None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:17.542805 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa912c1-128b-4217-ba9e-9256c0fbc656 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:17.543084 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa912c1-128b-4217-ba9e-9256c0fbc656 None admin] PATH_INFO: `/v3/groups` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:17.543751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa912c1-128b-4217-ba9e-9256c0fbc656 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:17.545746 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa912c1-128b-4217-ba9e-9256c0fbc656 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:17.545859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e147b29-7684-4d53-87f6-b286184436d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:17.550622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-6e147b29-7684-4d53-87f6-b286184436d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 06:59:17.550622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:17.550622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:17.550622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:17.550622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:17.550622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:17.550622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:17.550622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:17.550622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:17.550622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:17.550622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:17.550622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:17.550622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:17.550622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:17.550622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:17.550622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:17.551530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:17.551530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:17.551530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:17.551530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:17.551530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:17.551530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:17.551530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:17.551530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:17.551530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:17.551530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:17.551530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:17.551530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:17.551530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:17.551530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:17.551530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:17.551530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:17.552476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:17.552476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:17.552476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:17.552476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:17.552476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:17.552476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:17.552476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:17.552476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:17.552476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:17.552476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:17.552476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 06:59:17.552476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:17.552476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 170/327] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:17.553127 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e147b29-7684-4d53-87f6-b286184436d7 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:17.565842 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 158/328] 10.222.0.17 () {60 vars in 1134 bytes} [Tue Aug 23 06:59:17 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:59:17.568770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa912c1-128b-4217-ba9e-9256c0fbc656 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:17.677944 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e147b29-7684-4d53-87f6-b286184436d7 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:17.694644 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa912c1-128b-4217-ba9e-9256c0fbc656 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:17.725925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc9f33-c6ac-472e-9f67-3ef2dd4c375a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:17.726556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc9f33-c6ac-472e-9f67-3ef2dd4c375a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:17.727246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc9f33-c6ac-472e-9f67-3ef2dd4c375a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:17.727246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc9f33-c6ac-472e-9f67-3ef2dd4c375a None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:17.728334 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc9f33-c6ac-472e-9f67-3ef2dd4c375a None admin] RBAC: Adding query filter params (name=member) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:17.729370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc9f33-c6ac-472e-9f67-3ef2dd4c375a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:17.730049 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc9f33-c6ac-472e-9f67-3ef2dd4c375a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:17.732688 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b89aa5-1b5e-43d4-81a9-08f1c02874df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:17.733494 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b89aa5-1b5e-43d4-81a9-08f1c02874df None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:17.733915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b89aa5-1b5e-43d4-81a9-08f1c02874df None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:17.734193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b89aa5-1b5e-43d4-81a9-08f1c02874df None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:17.734880 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b89aa5-1b5e-43d4-81a9-08f1c02874df None admin] RBAC: Adding query filter params (name=reader) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:17.735182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b89aa5-1b5e-43d4-81a9-08f1c02874df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:17.736634 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b89aa5-1b5e-43d4-81a9-08f1c02874df None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:17.740773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 171/329] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:59:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:17.746184 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 159/330] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:59:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:17.747390 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc9f33-c6ac-472e-9f67-3ef2dd4c375a None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:17.751460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:17.752818 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b89aa5-1b5e-43d4-81a9-08f1c02874df None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:17.776291 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:17.872710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc9f33-c6ac-472e-9f67-3ef2dd4c375a None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:17.873288 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b89aa5-1b5e-43d4-81a9-08f1c02874df None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:17.915288 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1827c3f-a636-4412-b814-9208a360c109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:17.915837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70c139-4a2c-4176-a460-927b9c37e228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:17.916475 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70c139-4a2c-4176-a460-927b9c37e228 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:17.916751 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70c139-4a2c-4176-a460-927b9c37e228 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:17.917020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70c139-4a2c-4176-a460-927b9c37e228 None admin] PATH_INFO: `/v3/users/d157873e84564a86a03bc9899fbf6711` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:17.917531 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1827c3f-a636-4412-b814-9208a360c109 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:17.918018 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1827c3f-a636-4412-b814-9208a360c109 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:17.918146 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70c139-4a2c-4176-a460-927b9c37e228 None admin] RBAC: Authorizing `identity:get_user(user_id=d157873e84564a86a03bc9899fbf6711)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:17.918627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1827c3f-a636-4412-b814-9208a360c109 None admin] PATH_INFO: `/v3/users/7b05b363aae4464ead3a8674e2d53015` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:17.919633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70c139-4a2c-4176-a460-927b9c37e228 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:17.920430 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1827c3f-a636-4412-b814-9208a360c109 None admin] RBAC: Authorizing `identity:get_user(user_id=7b05b363aae4464ead3a8674e2d53015)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:17.921624 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 172/331] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:59:17 2022] GET /identity/v3/users/d157873e84564a86a03bc9899fbf6711 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:17.921888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1827c3f-a636-4412-b814-9208a360c109 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:17.923613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 160/332] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:59:17 2022] GET /identity/v3/users/7b05b363aae4464ead3a8674e2d53015 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:17.927532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70c139-4a2c-4176-a460-927b9c37e228 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:17.929840 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1827c3f-a636-4412-b814-9208a360c109 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:18.013772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:18.039919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:18.053062 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1827c3f-a636-4412-b814-9208a360c109 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:18.055772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70c139-4a2c-4176-a460-927b9c37e228 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:18.103459 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46139c2-c93b-454f-9ff2-a36564dcdc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:18.103663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d1b29-8f5a-47b1-999d-94c08c4a69e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:18.104230 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d1b29-8f5a-47b1-999d-94c08c4a69e9 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:18.104587 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d1b29-8f5a-47b1-999d-94c08c4a69e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:18.104973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46139c2-c93b-454f-9ff2-a36564dcdc88 None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:18.105410 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46139c2-c93b-454f-9ff2-a36564dcdc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:18.105509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d1b29-8f5a-47b1-999d-94c08c4a69e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:18.106003 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46139c2-c93b-454f-9ff2-a36564dcdc88 None admin] PATH_INFO: `/v3/system/users/7b05b363aae4464ead3a8674e2d53015/roles/ff4e7aa3dd7e4272a3cd1cafb32baa24` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:18.106293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d1b29-8f5a-47b1-999d-94c08c4a69e9 None admin] RBAC: Adding query filter params (role.id=7670c7c450b9404ba57e12743cb6c458, user.id=d157873e84564a86a03bc9899fbf6711, scope.system=all) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:18.106675 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d1b29-8f5a-47b1-999d-94c08c4a69e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:18.108124 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46139c2-c93b-454f-9ff2-a36564dcdc88 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7b05b363aae4464ead3a8674e2d53015, role_id=ff4e7aa3dd7e4272a3cd1cafb32baa24)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:18.108546 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d1b29-8f5a-47b1-999d-94c08c4a69e9 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:18.109973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46139c2-c93b-454f-9ff2-a36564dcdc88 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:18.126229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 161/333] 10.222.0.17 () {58 vars in 1334 bytes} [Tue Aug 23 06:59:17 2022] PUT /identity/v3/system/users/7b05b363aae4464ead3a8674e2d53015/roles/ff4e7aa3dd7e4272a3cd1cafb32baa24 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 06:59:18.129188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 173/334] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Aug 23 06:59:17 2022] GET /identity/v3/role_assignments?user.id=d157873e84564a86a03bc9899fbf6711&scope.system=all&role.id=7670c7c450b9404ba57e12743cb6c458 => generated 510 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:19.934482 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb75656-e8d5-4adf-9e85-440479c5243e None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:19.935544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb75656-e8d5-4adf-9e85-440479c5243e None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:19.936089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb75656-e8d5-4adf-9e85-440479c5243e None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:19.937882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 162/335] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:19.945069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebd099-1b92-46bc-816c-f8f1aed92077 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:19.946331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebd099-1b92-46bc-816c-f8f1aed92077 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:19.946824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebd099-1b92-46bc-816c-f8f1aed92077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:19.968500 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-a0ebd099-1b92-46bc-816c-f8f1aed92077 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:19.970039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a0ebd099-1b92-46bc-816c-f8f1aed92077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:20.060165 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 174/336] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:59:20.171781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb59c1-fcbf-42bd-8abc-780523e5cd09 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:20.172477 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb59c1-fcbf-42bd-8abc-780523e5cd09 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:20.172951 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb59c1-fcbf-42bd-8abc-780523e5cd09 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:20.174744 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 163/337] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:20.189373 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cab0ca-84cf-4c0a-9393-6c1eb0a550fd None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:20.189640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cab0ca-84cf-4c0a-9393-6c1eb0a550fd None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:20.189851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cab0ca-84cf-4c0a-9393-6c1eb0a550fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:20.212487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-47cab0ca-84cf-4c0a-9393-6c1eb0a550fd None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:20.213688 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-47cab0ca-84cf-4c0a-9393-6c1eb0a550fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:20.283519 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:20.298287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 175/338] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:20.314816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349970fd-5d19-49d1-a810-74b3b7a5873b None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:20.315126 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349970fd-5d19-49d1-a810-74b3b7a5873b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:20.315365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349970fd-5d19-49d1-a810-74b3b7a5873b None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:20.316890 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 164/339] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:20.323263 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb66c99-38d3-4802-99df-fd690293972b None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:20.323584 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb66c99-38d3-4802-99df-fd690293972b None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:20.323855 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb66c99-38d3-4802-99df-fd690293972b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:20.343889 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-8cb66c99-38d3-4802-99df-fd690293972b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:20.345075 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8cb66c99-38d3-4802-99df-fd690293972b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:20.429069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 176/340] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:20.490810 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349970fd-5d19-49d1-a810-74b3b7a5873b None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:20.562298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1f335-0402-4fa4-b063-2ef26bc12e5b None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:20.562929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1f335-0402-4fa4-b063-2ef26bc12e5b None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:20.563406 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1f335-0402-4fa4-b063-2ef26bc12e5b None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:20.565033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 177/341] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:59:20.571820 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0415f0-b820-4892-92c6-f45225fd291c None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:20.572375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0415f0-b820-4892-92c6-f45225fd291c None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:20.572858 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0415f0-b820-4892-92c6-f45225fd291c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:20.593645 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-6e0415f0-b820-4892-92c6-f45225fd291c None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:20.595197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6e0415f0-b820-4892-92c6-f45225fd291c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:20.608965 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349970fd-5d19-49d1-a810-74b3b7a5873b None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:20.672941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e2bfc-9814-4c96-aa17-c2fac1544fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:20.673883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e2bfc-9814-4c96-aa17-c2fac1544fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:20.674348 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e2bfc-9814-4c96-aa17-c2fac1544fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:20.674794 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e2bfc-9814-4c96-aa17-c2fac1544fe0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:20.706660 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4e2bfc-9814-4c96-aa17-c2fac1544fe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:20.719813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-8c4e2bfc-9814-4c96-aa17-c2fac1544fe0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:59:20.719813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:20.719813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:20.719813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:20.719813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:20.719813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:20.719813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:20.719813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:20.719813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:20.719813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:20.719813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:20.719813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:20.719813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:20.719813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:20.719813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:20.719813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:20.720756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:20.720756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:20.720756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:20.720756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:20.720756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:20.720756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:20.720756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:20.720756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:20.720756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:20.720756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:20.720756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:20.720756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:20.720756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:20.720756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:20.720756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:20.720756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:20.721552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:20.721552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:20.721552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:20.721552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:20.721552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:20.721552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:20.721552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:20.721552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:20.721552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:20.721552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:20.721552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:59:20.721552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:20.724154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 165/342] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 06:59:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:20.725819 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 178/343] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:59:20.729999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e2bfc-9814-4c96-aa17-c2fac1544fe0 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:20.747954 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:20.834235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0415f0-b820-4892-92c6-f45225fd291c None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:20.876622 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e2bfc-9814-4c96-aa17-c2fac1544fe0 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:20.915471 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:20.935522 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230807b-88f8-4063-b657-899761cc010a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:20.936256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230807b-88f8-4063-b657-899761cc010a None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:20.936594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230807b-88f8-4063-b657-899761cc010a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:20.936875 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230807b-88f8-4063-b657-899761cc010a None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:20.937603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230807b-88f8-4063-b657-899761cc010a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:20.937914 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230807b-88f8-4063-b657-899761cc010a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:20.939312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230807b-88f8-4063-b657-899761cc010a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:20.948404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 166/344] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:20.949343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230807b-88f8-4063-b657-899761cc010a None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:20.994924 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0415f0-b820-4892-92c6-f45225fd291c None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:21.049479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05242ed-8018-4914-af93-91111f7e5335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:21.049691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:21.050307 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05242ed-8018-4914-af93-91111f7e5335 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:21.050685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05242ed-8018-4914-af93-91111f7e5335 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:21.051151 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05242ed-8018-4914-af93-91111f7e5335 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:21.068584 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05242ed-8018-4914-af93-91111f7e5335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:21.069623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230807b-88f8-4063-b657-899761cc010a None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:21.073976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-f05242ed-8018-4914-af93-91111f7e5335 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 06:59:21.073976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:21.073976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:21.073976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:21.073976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:21.073976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:21.073976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:21.073976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:21.073976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:21.073976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:21.073976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:21.073976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:21.073976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:21.073976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:21.073976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:21.073976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:21.074851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:21.074851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:21.074851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:21.074851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:21.074851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:21.074851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:21.074851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:21.074851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:21.074851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:21.074851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:21.074851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:21.074851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:21.074851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:21.074851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:21.074851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:21.074851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:21.075676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:21.075676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:21.075676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:21.075676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:21.075676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:21.075676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:21.075676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:21.075676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:21.075676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:21.075676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:21.075676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 06:59:21.075676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:21.079584 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 179/345] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:21.080901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05242ed-8018-4914-af93-91111f7e5335 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:21.111392 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bc84e-4c63-4794-9db4-f8f8c9b5097b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:21.112479 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bc84e-4c63-4794-9db4-f8f8c9b5097b None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:21.113012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bc84e-4c63-4794-9db4-f8f8c9b5097b None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:21.113481 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bc84e-4c63-4794-9db4-f8f8c9b5097b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:21.125181 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199bc84e-4c63-4794-9db4-f8f8c9b5097b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:21.130516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-199bc84e-4c63-4794-9db4-f8f8c9b5097b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:21.130516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:21.130516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:21.130516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:21.130516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:21.130516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:21.130516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:21.130516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:21.130516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:21.130516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:21.130516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:21.130516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:21.130516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:21.130516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:21.130516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:21.130516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:21.132652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:21.132652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:21.132652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:21.132652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:21.132652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:21.132652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:21.132652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:21.132652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:21.132652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:21.132652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:21.132652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:21.132652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:21.132652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:21.132652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:21.132652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:21.132652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:21.133500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:21.133500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:21.133500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:21.133500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:21.133500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:21.133500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:21.133500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:21.133500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:21.133500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:21.133500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:21.133500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:21.133500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:21.136019 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 167/346] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:21.137089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bc84e-4c63-4794-9db4-f8f8c9b5097b None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:21.193492 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05242ed-8018-4914-af93-91111f7e5335 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:21.207791 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:21.232559 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c783a5-d0b1-462e-914b-f835ed9b223b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:21.232762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c783a5-d0b1-462e-914b-f835ed9b223b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:21.233205 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c783a5-d0b1-462e-914b-f835ed9b223b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:21.233595 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c783a5-d0b1-462e-914b-f835ed9b223b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:21.255010 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bc84e-4c63-4794-9db4-f8f8c9b5097b None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:21.297522 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:21.304900 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a756d0b-07b0-42ff-b1ba-ad816f7be9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:21.305788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a756d0b-07b0-42ff-b1ba-ad816f7be9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:21.306272 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a756d0b-07b0-42ff-b1ba-ad816f7be9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:21.307382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a756d0b-07b0-42ff-b1ba-ad816f7be9b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:21.308630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a756d0b-07b0-42ff-b1ba-ad816f7be9b5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:21.309169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a756d0b-07b0-42ff-b1ba-ad816f7be9b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:21.311094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a756d0b-07b0-42ff-b1ba-ad816f7be9b5 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:21.317070 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c783a5-d0b1-462e-914b-f835ed9b223b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:21.318553 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c783a5-d0b1-462e-914b-f835ed9b223b None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:21.323641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 168/347] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:59:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:21.324703 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a756d0b-07b0-42ff-b1ba-ad816f7be9b5 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:21.437936 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 180/348] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:21.438835 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c783a5-d0b1-462e-914b-f835ed9b223b None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:21.456069 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a756d0b-07b0-42ff-b1ba-ad816f7be9b5 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:21.457081 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:21.496688 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8b748-e5cd-4a18-ba09-bc1c64e3a9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:21.497497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8b748-e5cd-4a18-ba09-bc1c64e3a9ef None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:21.497875 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8b748-e5cd-4a18-ba09-bc1c64e3a9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:21.498266 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8b748-e5cd-4a18-ba09-bc1c64e3a9ef None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:21.499175 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8b748-e5cd-4a18-ba09-bc1c64e3a9ef None admin] RBAC: Adding query filter params (name=member) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:21.499652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8b748-e5cd-4a18-ba09-bc1c64e3a9ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:21.501297 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8b748-e5cd-4a18-ba09-bc1c64e3a9ef None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:21.511570 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 169/349] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:59:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:21.512218 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8b748-e5cd-4a18-ba09-bc1c64e3a9ef None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:21.561076 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c783a5-d0b1-462e-914b-f835ed9b223b None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:21.575863 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:21.600247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b3e2e-4ba3-491d-94cf-110ce4708cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:21.600890 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b3e2e-4ba3-491d-94cf-110ce4708cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:21.601163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b3e2e-4ba3-491d-94cf-110ce4708cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:21.601436 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b3e2e-4ba3-491d-94cf-110ce4708cc5 None admin] PATH_INFO: `/v3/users/7b05b363aae4464ead3a8674e2d53015` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:21.604158 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b3e2e-4ba3-491d-94cf-110ce4708cc5 None admin] RBAC: Authorizing `identity:get_user(user_id=7b05b363aae4464ead3a8674e2d53015)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:21.605606 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b3e2e-4ba3-491d-94cf-110ce4708cc5 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:21.611534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 181/350] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 06:59:21 2022] GET /identity/v3/users/7b05b363aae4464ead3a8674e2d53015 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:21.612441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b3e2e-4ba3-491d-94cf-110ce4708cc5 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:21.624435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8b748-e5cd-4a18-ba09-bc1c64e3a9ef None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:21.665702 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3565cf-bfdc-49cc-9022-fd94474956a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:21.666449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3565cf-bfdc-49cc-9022-fd94474956a2 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:21.666737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3565cf-bfdc-49cc-9022-fd94474956a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:21.667011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3565cf-bfdc-49cc-9022-fd94474956a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:21.680934 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3565cf-bfdc-49cc-9022-fd94474956a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:21.686741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-6b3565cf-bfdc-49cc-9022-fd94474956a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:21.686741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:21.686741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:21.686741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:21.686741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:21.686741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:21.686741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:21.686741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:21.686741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:21.686741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:21.686741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:21.686741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:21.686741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:21.686741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:21.686741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:21.686741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:21.687771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:21.687771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:21.687771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:21.687771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:21.687771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:21.687771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:21.687771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:21.687771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:21.687771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:21.687771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:21.687771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:21.687771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:21.687771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:21.687771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:21.687771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:21.687771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:21.688557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:21.688557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:21.688557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:21.688557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:21.688557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:21.688557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:21.688557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:21.688557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:21.688557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:21.688557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:21.688557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:21.688557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:21.688557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 170/351] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:21.689213 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3565cf-bfdc-49cc-9022-fd94474956a2 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:21.700871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:21.725588 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b3e2e-4ba3-491d-94cf-110ce4708cc5 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:21.769594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb782b3-85f6-4cfd-a1e4-d3e151339e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:21.770390 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb782b3-85f6-4cfd-a1e4-d3e151339e64 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:21.770665 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb782b3-85f6-4cfd-a1e4-d3e151339e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:21.771029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb782b3-85f6-4cfd-a1e4-d3e151339e64 None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:21.772520 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb782b3-85f6-4cfd-a1e4-d3e151339e64 None admin] RBAC: Authorizing `identity:get_project(project_id=92ca046a717e4adab11f7371ad3cc888)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:21.774892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb782b3-85f6-4cfd-a1e4-d3e151339e64 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:21.778049 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 182/352] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:21 2022] GET /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:21.778871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb782b3-85f6-4cfd-a1e4-d3e151339e64 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:21.817978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3565cf-bfdc-49cc-9022-fd94474956a2 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:21.837209 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:21.861183 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae87717-9176-46fc-b53c-7d46f97d38bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:21.861804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae87717-9176-46fc-b53c-7d46f97d38bd None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:21.862081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae87717-9176-46fc-b53c-7d46f97d38bd None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:21.862353 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae87717-9176-46fc-b53c-7d46f97d38bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:21.917138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb782b3-85f6-4cfd-a1e4-d3e151339e64 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:21.937408 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae87717-9176-46fc-b53c-7d46f97d38bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:21.938954 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae87717-9176-46fc-b53c-7d46f97d38bd None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:21.960382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c54e27-2f6a-419f-8b54-a4078a50cbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:21.961153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c54e27-2f6a-419f-8b54-a4078a50cbd1 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:21.961718 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c54e27-2f6a-419f-8b54-a4078a50cbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:21.961995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c54e27-2f6a-419f-8b54-a4078a50cbd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:21.962734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c54e27-2f6a-419f-8b54-a4078a50cbd1 None admin] RBAC: Adding query filter params (role.id=ff4e7aa3dd7e4272a3cd1cafb32baa24, user.id=7b05b363aae4464ead3a8674e2d53015, scope.system=all) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:21.963038 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c54e27-2f6a-419f-8b54-a4078a50cbd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:21.964947 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c54e27-2f6a-419f-8b54-a4078a50cbd1 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:21.980899 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 183/353] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Aug 23 06:59:21 2022] GET /identity/v3/role_assignments?user.id=7b05b363aae4464ead3a8674e2d53015&scope.system=all&role.id=ff4e7aa3dd7e4272a3cd1cafb32baa24 => generated 510 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:21.981756 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c54e27-2f6a-419f-8b54-a4078a50cbd1 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:21.991895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:22.092447 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 171/354] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:22.093312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae87717-9176-46fc-b53c-7d46f97d38bd None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:22.117105 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:22.132538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c54e27-2f6a-419f-8b54-a4078a50cbd1 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:22.181536 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e74b4-cdcb-4216-a7ad-eadbcfe77775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:22.182158 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e74b4-cdcb-4216-a7ad-eadbcfe77775 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:22.182433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e74b4-cdcb-4216-a7ad-eadbcfe77775 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:22.182911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e74b4-cdcb-4216-a7ad-eadbcfe77775 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:22.184289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e74b4-cdcb-4216-a7ad-eadbcfe77775 None admin] RBAC: Adding query filter params (name=member) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:22.184596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e74b4-cdcb-4216-a7ad-eadbcfe77775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:22.187520 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e74b4-cdcb-4216-a7ad-eadbcfe77775 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:22.195254 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 184/355] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:59:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:22.198439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6036c0-9785-45c9-9c2e-a246e7cd5912 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:22.198721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6036c0-9785-45c9-9c2e-a246e7cd5912 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:22.198991 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6036c0-9785-45c9-9c2e-a246e7cd5912 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:22.201135 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 185/356] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:22.201271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6036c0-9785-45c9-9c2e-a246e7cd5912 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:22.240314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae87717-9176-46fc-b53c-7d46f97d38bd None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:22.290157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1eb22-fd71-41a2-9c1d-8644c216e44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:22.290806 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1eb22-fd71-41a2-9c1d-8644c216e44c None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:22.291086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1eb22-fd71-41a2-9c1d-8644c216e44c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:22.291360 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1eb22-fd71-41a2-9c1d-8644c216e44c None admin] PATH_INFO: `/v3/groups/8722ee6835904279adfb53a5649018b7` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:22.294801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:22.298548 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1eb22-fd71-41a2-9c1d-8644c216e44c None admin] RBAC: Authorizing `identity:get_group(group_id=8722ee6835904279adfb53a5649018b7)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:22.300100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1eb22-fd71-41a2-9c1d-8644c216e44c None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:22.301826 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 172/357] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 23 06:59:22 2022] GET /identity/v3/groups/8722ee6835904279adfb53a5649018b7 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:22.302596 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1eb22-fd71-41a2-9c1d-8644c216e44c None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:22.334427 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6036c0-9785-45c9-9c2e-a246e7cd5912 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:22.375379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec953563-62d6-405f-a82d-f91d007a726d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:22.376076 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec953563-62d6-405f-a82d-f91d007a726d None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:22.376349 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec953563-62d6-405f-a82d-f91d007a726d None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:22.376617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec953563-62d6-405f-a82d-f91d007a726d None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:22.377724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec953563-62d6-405f-a82d-f91d007a726d None admin] RBAC: Authorizing `identity:get_project(project_id=92ca046a717e4adab11f7371ad3cc888)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:22.379155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec953563-62d6-405f-a82d-f91d007a726d None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:22.381147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 186/358] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:22 2022] GET /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:22.382279 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cafb71-1f0d-46be-b804-59a6ac399eb2 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:22.382555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cafb71-1f0d-46be-b804-59a6ac399eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:22.382823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cafb71-1f0d-46be-b804-59a6ac399eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:22.401701 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:22.401951 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-e6cafb71-1f0d-46be-b804-59a6ac399eb2 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:22.403451 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e6cafb71-1f0d-46be-b804-59a6ac399eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:22.424695 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1eb22-fd71-41a2-9c1d-8644c216e44c None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:22.466262 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fa37f-3a12-46f2-ad20-decc827a385e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:22.466926 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fa37f-3a12-46f2-ad20-decc827a385e None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:22.467202 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fa37f-3a12-46f2-ad20-decc827a385e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:22.472590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fa37f-3a12-46f2-ad20-decc827a385e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:22.501508 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 187/359] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:22.505652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f5761-97b9-471b-9f76-4a34a677cd63 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:22.505652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f5761-97b9-471b-9f76-4a34a677cd63 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:22.505652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f5761-97b9-471b-9f76-4a34a677cd63 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:22.506078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 188/360] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:22.506866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f5761-97b9-471b-9f76-4a34a677cd63 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:22.563743 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642fa37f-3a12-46f2-ad20-decc827a385e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:22.565232 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642fa37f-3a12-46f2-ad20-decc827a385e None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:22.595192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:22.649344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f5761-97b9-471b-9f76-4a34a677cd63 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:22.669721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:22.699969 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 173/361] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:22.701139 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fa37f-3a12-46f2-ad20-decc827a385e None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:22.701435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6826d3-22e5-4036-80e2-fe5f175ec385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:22.702113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6826d3-22e5-4036-80e2-fe5f175ec385 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:22.702393 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6826d3-22e5-4036-80e2-fe5f175ec385 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:22.702667 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6826d3-22e5-4036-80e2-fe5f175ec385 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:22.703406 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6826d3-22e5-4036-80e2-fe5f175ec385 None admin] RBAC: Adding query filter params (group.id=8722ee6835904279adfb53a5649018b7, role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7, scope.project.id=92ca046a717e4adab11f7371ad3cc888) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:22.703743 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6826d3-22e5-4036-80e2-fe5f175ec385 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:22.705454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6826d3-22e5-4036-80e2-fe5f175ec385 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:22.712608 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 189/362] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 23 06:59:22 2022] GET /identity/v3/role_assignments?group.id=8722ee6835904279adfb53a5649018b7&scope.project.id=92ca046a717e4adab11f7371ad3cc888&role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:22.713833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92264ad-57e5-41ea-9f60-a0b933b3072f None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:22.714119 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92264ad-57e5-41ea-9f60-a0b933b3072f None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:22.714390 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92264ad-57e5-41ea-9f60-a0b933b3072f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:22.740309 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-c92264ad-57e5-41ea-9f60-a0b933b3072f None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:22.741524 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c92264ad-57e5-41ea-9f60-a0b933b3072f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:22.827691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fa37f-3a12-46f2-ad20-decc827a385e None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:22.832342 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 190/363] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:22.833133 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92264ad-57e5-41ea-9f60-a0b933b3072f None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:22.884133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5db162-a56f-48e6-8fee-7a422f87656b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:22.885840 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5db162-a56f-48e6-8fee-7a422f87656b None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:22.886047 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5db162-a56f-48e6-8fee-7a422f87656b None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:22.886322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5db162-a56f-48e6-8fee-7a422f87656b None admin] PATH_INFO: `/v3/groups/8722ee6835904279adfb53a5649018b7` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:22.887498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5db162-a56f-48e6-8fee-7a422f87656b None admin] RBAC: Authorizing `identity:get_group(group_id=8722ee6835904279adfb53a5649018b7)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:22.888985 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5db162-a56f-48e6-8fee-7a422f87656b None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:22.892158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 174/364] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 23 06:59:22 2022] GET /identity/v3/groups/8722ee6835904279adfb53a5649018b7 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:22.893039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5db162-a56f-48e6-8fee-7a422f87656b None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:22.915382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:22.968599 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:22.970460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92264ad-57e5-41ea-9f60-a0b933b3072f None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:23.019455 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78a108-64b3-456b-a251-4ed51d881140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:23.020337 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78a108-64b3-456b-a251-4ed51d881140 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:23.020619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78a108-64b3-456b-a251-4ed51d881140 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:23.020893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78a108-64b3-456b-a251-4ed51d881140 None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:23.022446 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78a108-64b3-456b-a251-4ed51d881140 None admin] RBAC: Authorizing `identity:get_project(project_id=e594e988024e47b4a48b43d8f828c605)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:23.024407 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78a108-64b3-456b-a251-4ed51d881140 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:23.027281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 191/365] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:22 2022] GET /identity/v3/projects/e594e988024e47b4a48b43d8f828c605 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:23.028617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78a108-64b3-456b-a251-4ed51d881140 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:23.033783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5db162-a56f-48e6-8fee-7a422f87656b None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:23.074306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714bb13-7e91-4815-902d-39f8e114bf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:23.075367 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714bb13-7e91-4815-902d-39f8e114bf92 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:23.075904 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714bb13-7e91-4815-902d-39f8e114bf92 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:23.076423 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714bb13-7e91-4815-902d-39f8e114bf92 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:23.089039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714bb13-7e91-4815-902d-39f8e114bf92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:23.094432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-a714bb13-7e91-4815-902d-39f8e114bf92 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:59:23.094432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:23.094432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:23.094432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:23.094432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:23.094432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:23.094432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:23.094432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:23.094432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:23.094432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:23.094432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:23.094432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:23.094432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:23.094432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:23.094432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:23.094432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:23.095343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:23.095343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:23.095343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:23.095343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:23.095343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:23.095343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:23.095343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:23.095343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:23.095343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:23.095343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:23.095343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:23.095343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:23.095343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:23.095343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:23.095343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:23.095343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:23.096182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:23.096182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:23.096182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:23.096182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:23.096182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:23.096182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:23.096182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:23.096182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:23.096182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:23.096182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:23.096182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:59:23.096182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:23.098545 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 175/366] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 06:59:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:23.099592 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714bb13-7e91-4815-902d-39f8e114bf92 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:23.160789 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78a108-64b3-456b-a251-4ed51d881140 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:23.198431 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:23.216969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a57d55-32a6-49de-a547-811430b218be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:23.217195 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a57d55-32a6-49de-a547-811430b218be None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:23.220419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a57d55-32a6-49de-a547-811430b218be None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:23.220419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a57d55-32a6-49de-a547-811430b218be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:23.220419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a57d55-32a6-49de-a547-811430b218be None admin] RBAC: Adding query filter params (group.id=8722ee6835904279adfb53a5649018b7, role.id=7670c7c450b9404ba57e12743cb6c458, scope.project.id=92ca046a717e4adab11f7371ad3cc888) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:23.220419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a57d55-32a6-49de-a547-811430b218be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:23.221161 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a57d55-32a6-49de-a547-811430b218be None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:23.227467 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 192/367] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 23 06:59:23 2022] GET /identity/v3/role_assignments?group.id=8722ee6835904279adfb53a5649018b7&scope.project.id=92ca046a717e4adab11f7371ad3cc888&role.id=7670c7c450b9404ba57e12743cb6c458 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:23.228296 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a57d55-32a6-49de-a547-811430b218be None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:23.230937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714bb13-7e91-4815-902d-39f8e114bf92 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:23.243830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:23.277416 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffca6e8-a11b-4e12-83db-4712c7fa588a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:23.278074 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffca6e8-a11b-4e12-83db-4712c7fa588a None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:23.278352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffca6e8-a11b-4e12-83db-4712c7fa588a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:23.278625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffca6e8-a11b-4e12-83db-4712c7fa588a None admin] PATH_INFO: `/v3/groups/8722ee6835904279adfb53a5649018b7` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:23.279978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffca6e8-a11b-4e12-83db-4712c7fa588a None admin] RBAC: Authorizing `identity:get_group(group_id=8722ee6835904279adfb53a5649018b7)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:23.283201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffca6e8-a11b-4e12-83db-4712c7fa588a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:23.284935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 176/368] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 23 06:59:23 2022] GET /identity/v3/groups/8722ee6835904279adfb53a5649018b7 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:23.285799 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffca6e8-a11b-4e12-83db-4712c7fa588a None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:23.378353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a57d55-32a6-49de-a547-811430b218be None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:23.425606 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93a937-457c-4545-a5a9-b4b0060ffbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:23.426235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93a937-457c-4545-a5a9-b4b0060ffbab None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:23.426515 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93a937-457c-4545-a5a9-b4b0060ffbab None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:23.426789 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93a937-457c-4545-a5a9-b4b0060ffbab None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:23.427620 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93a937-457c-4545-a5a9-b4b0060ffbab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:23.427929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93a937-457c-4545-a5a9-b4b0060ffbab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:23.429315 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93a937-457c-4545-a5a9-b4b0060ffbab None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:23.430512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffca6e8-a11b-4e12-83db-4712c7fa588a None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:23.440031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 193/369] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:23.441095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93a937-457c-4545-a5a9-b4b0060ffbab None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:23.480724 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7eab8-c7c1-480d-9418-4c13fbf81c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:23.481313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7eab8-c7c1-480d-9418-4c13fbf81c3e None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:23.481865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7eab8-c7c1-480d-9418-4c13fbf81c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:23.482142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7eab8-c7c1-480d-9418-4c13fbf81c3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:23.496715 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7eab8-c7c1-480d-9418-4c13fbf81c3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:23.502774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-b5f7eab8-c7c1-480d-9418-4c13fbf81c3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:59:23.502774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:23.502774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:23.502774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:23.502774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:23.502774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:23.502774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:23.502774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:23.502774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:23.502774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:23.502774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:23.502774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:23.502774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:23.502774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:23.502774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:23.502774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:23.503737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:23.503737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:23.503737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:23.503737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:23.503737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:23.503737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:23.503737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:23.503737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:23.503737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:23.503737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:23.503737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:23.503737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:23.503737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:23.503737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:23.503737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:23.503737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:23.504574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:23.504574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:23.504574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:23.504574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:23.504574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:23.504574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:23.504574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:23.504574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:23.504574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:23.504574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:23.504574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:59:23.504574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:23.505182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:23.506650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 177/370] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 06:59:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:23.507096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7eab8-c7c1-480d-9418-4c13fbf81c3e None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:23.562225 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:23.564683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93a937-457c-4545-a5a9-b4b0060ffbab None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:23.612491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8216416-5fcf-4497-a822-180138e29068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:23.613150 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8216416-5fcf-4497-a822-180138e29068 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:23.613433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8216416-5fcf-4497-a822-180138e29068 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:23.614126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8216416-5fcf-4497-a822-180138e29068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:23.614873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8216416-5fcf-4497-a822-180138e29068 None admin] RBAC: Adding query filter params (group.id=8722ee6835904279adfb53a5649018b7, role.id=7670c7c450b9404ba57e12743cb6c458, scope.project.id=e594e988024e47b4a48b43d8f828c605) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:23.615183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8216416-5fcf-4497-a822-180138e29068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:23.617542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8216416-5fcf-4497-a822-180138e29068 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:23.626153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 194/371] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 23 06:59:23 2022] GET /identity/v3/role_assignments?group.id=8722ee6835904279adfb53a5649018b7&scope.project.id=e594e988024e47b4a48b43d8f828c605&role.id=7670c7c450b9404ba57e12743cb6c458 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:23.626998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8216416-5fcf-4497-a822-180138e29068 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:23.643624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7eab8-c7c1-480d-9418-4c13fbf81c3e None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:23.721603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed75c06-07af-46ce-a3f1-305c29cdfb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:23.722484 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed75c06-07af-46ce-a3f1-305c29cdfb79 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:23.722766 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed75c06-07af-46ce-a3f1-305c29cdfb79 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:23.723035 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed75c06-07af-46ce-a3f1-305c29cdfb79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:23.761393 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8216416-5fcf-4497-a822-180138e29068 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:23.785318 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:23.807974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4680f5b-0cf3-424a-95bc-1a566f6a70a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:23.808947 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4680f5b-0cf3-424a-95bc-1a566f6a70a0 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:23.809445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4680f5b-0cf3-424a-95bc-1a566f6a70a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:23.809912 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4680f5b-0cf3-424a-95bc-1a566f6a70a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:23.810815 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4680f5b-0cf3-424a-95bc-1a566f6a70a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:23.811311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4680f5b-0cf3-424a-95bc-1a566f6a70a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:23.813688 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4680f5b-0cf3-424a-95bc-1a566f6a70a0 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:23.817800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed75c06-07af-46ce-a3f1-305c29cdfb79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:23.819288 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed75c06-07af-46ce-a3f1-305c29cdfb79 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:23.825446 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 195/372] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:23.832488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4680f5b-0cf3-424a-95bc-1a566f6a70a0 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:23.861640 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:23.941064 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 178/373] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:23.942400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed75c06-07af-46ce-a3f1-305c29cdfb79 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:24.048172 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed75c06-07af-46ce-a3f1-305c29cdfb79 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:24.068653 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4680f5b-0cf3-424a-95bc-1a566f6a70a0 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:24.087656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7a19c-a9e6-43f7-9e8c-f8ffad84b0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:24.088412 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7a19c-a9e6-43f7-9e8c-f8ffad84b0b9 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:24.088762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7a19c-a9e6-43f7-9e8c-f8ffad84b0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:24.089106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7a19c-a9e6-43f7-9e8c-f8ffad84b0b9 None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:24.092243 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc7a19c-a9e6-43f7-9e8c-f8ffad84b0b9 None admin] RBAC: Authorizing `identity:get_project(project_id=e594e988024e47b4a48b43d8f828c605)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:24.094060 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc7a19c-a9e6-43f7-9e8c-f8ffad84b0b9 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:24.099431 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 179/374] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:23 2022] GET /identity/v3/projects/e594e988024e47b4a48b43d8f828c605 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:24.102095 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7a19c-a9e6-43f7-9e8c-f8ffad84b0b9 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:24.111776 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:24.138783 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2f1c3-996c-4aeb-b3b3-88f15ad9f0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:24.139737 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2f1c3-996c-4aeb-b3b3-88f15ad9f0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:24.140622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2f1c3-996c-4aeb-b3b3-88f15ad9f0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:24.141039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2f1c3-996c-4aeb-b3b3-88f15ad9f0a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:24.212731 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7a19c-a9e6-43f7-9e8c-f8ffad84b0b9 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:24.250620 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b8cf7-836d-4c7a-a009-7e7cbed14b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:24.252716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b8cf7-836d-4c7a-a009-7e7cbed14b51 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:24.252716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b8cf7-836d-4c7a-a009-7e7cbed14b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:24.252716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b8cf7-836d-4c7a-a009-7e7cbed14b51 None admin] PATH_INFO: `/v3/groups/8722ee6835904279adfb53a5649018b7` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:24.253747 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b8cf7-836d-4c7a-a009-7e7cbed14b51 None admin] RBAC: Authorizing `identity:get_group(group_id=8722ee6835904279adfb53a5649018b7)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:24.256493 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b8cf7-836d-4c7a-a009-7e7cbed14b51 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:24.259490 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 180/375] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 23 06:59:24 2022] GET /identity/v3/groups/8722ee6835904279adfb53a5649018b7 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:24.266011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b8cf7-836d-4c7a-a009-7e7cbed14b51 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:24.291779 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:24.321339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2f1c3-996c-4aeb-b3b3-88f15ad9f0a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:24.323342 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2f1c3-996c-4aeb-b3b3-88f15ad9f0a0 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:24.343855 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:24.380784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b8cf7-836d-4c7a-a009-7e7cbed14b51 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:24.419371 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6f8f2-8e2d-4a4c-a922-58698a440e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:24.420398 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6f8f2-8e2d-4a4c-a922-58698a440e51 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:24.420505 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6f8f2-8e2d-4a4c-a922-58698a440e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:24.420908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6f8f2-8e2d-4a4c-a922-58698a440e51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:24.421806 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6f8f2-8e2d-4a4c-a922-58698a440e51 None admin] RBAC: Adding query filter params (group.id=8722ee6835904279adfb53a5649018b7, role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7, scope.project.id=e594e988024e47b4a48b43d8f828c605) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:24.423052 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6f8f2-8e2d-4a4c-a922-58698a440e51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:24.424647 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6f8f2-8e2d-4a4c-a922-58698a440e51 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:24.431355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 196/376] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:24.431544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 181/377] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 23 06:59:24 2022] GET /identity/v3/role_assignments?group.id=8722ee6835904279adfb53a5649018b7&scope.project.id=e594e988024e47b4a48b43d8f828c605&role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:24.437803 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6f8f2-8e2d-4a4c-a922-58698a440e51 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:24.576757 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6f8f2-8e2d-4a4c-a922-58698a440e51 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:24.617189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdda77-01e2-4cfa-999b-ea00ee808b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:24.617897 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdda77-01e2-4cfa-999b-ea00ee808b4e None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:24.618182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdda77-01e2-4cfa-999b-ea00ee808b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:24.618456 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdda77-01e2-4cfa-999b-ea00ee808b4e None admin] PATH_INFO: `/v3/projects/c7014afbcdaa44f68b1b75e344808f41` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:24.619717 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdda77-01e2-4cfa-999b-ea00ee808b4e None admin] RBAC: Authorizing `identity:get_project(project_id=c7014afbcdaa44f68b1b75e344808f41)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:24.621244 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdda77-01e2-4cfa-999b-ea00ee808b4e None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:24.623317 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 182/378] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:24 2022] GET /identity/v3/projects/c7014afbcdaa44f68b1b75e344808f41 => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:24.632761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2f1c3-996c-4aeb-b3b3-88f15ad9f0a0 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:24.733576 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:24.761144 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2f1c3-996c-4aeb-b3b3-88f15ad9f0a0 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:24.795111 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9d286-36f7-45df-808d-2b0a81712b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:24.795776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9d286-36f7-45df-808d-2b0a81712b26 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:24.796273 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9d286-36f7-45df-808d-2b0a81712b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:24.796376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9d286-36f7-45df-808d-2b0a81712b26 None admin] PATH_INFO: `/v3/groups/3a712ed666f84aecbb18f75796c91ad3` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:24.802621 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9d286-36f7-45df-808d-2b0a81712b26 None admin] RBAC: Authorizing `identity:get_group(group_id=3a712ed666f84aecbb18f75796c91ad3)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:24.804104 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9d286-36f7-45df-808d-2b0a81712b26 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:24.806071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 197/379] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 23 06:59:24 2022] GET /identity/v3/groups/3a712ed666f84aecbb18f75796c91ad3 => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:24.811944 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdda77-01e2-4cfa-999b-ea00ee808b4e None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:24.819110 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:24.953696 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdda77-01e2-4cfa-999b-ea00ee808b4e None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:24.997333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ee6cc-6bd1-4bac-a493-02f5d7988adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:24.998006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ee6cc-6bd1-4bac-a493-02f5d7988adc None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:24.998284 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ee6cc-6bd1-4bac-a493-02f5d7988adc None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:24.998612 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ee6cc-6bd1-4bac-a493-02f5d7988adc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:24.999489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ee6cc-6bd1-4bac-a493-02f5d7988adc None admin] RBAC: Adding query filter params (group.id=3a712ed666f84aecbb18f75796c91ad3, role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.project.id=c7014afbcdaa44f68b1b75e344808f41) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:24.999798 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ee6cc-6bd1-4bac-a493-02f5d7988adc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:25.001265 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ee6cc-6bd1-4bac-a493-02f5d7988adc None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:25.007626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 183/380] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 23 06:59:24 2022] GET /identity/v3/role_assignments?group.id=3a712ed666f84aecbb18f75796c91ad3&scope.project.id=c7014afbcdaa44f68b1b75e344808f41&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:25.063280 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d63f0-5d15-444f-815b-a368f63e123a None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:25.063618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d63f0-5d15-444f-815b-a368f63e123a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:25.063856 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d63f0-5d15-444f-815b-a368f63e123a None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:25.065588 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 198/381] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:25.072308 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a090da1-2324-4d3c-b98a-c041a81653c0 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:25.072778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a090da1-2324-4d3c-b98a-c041a81653c0 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:25.072851 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a090da1-2324-4d3c-b98a-c041a81653c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:25.095393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-5a090da1-2324-4d3c-b98a-c041a81653c0 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:25.096680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-5a090da1-2324-4d3c-b98a-c041a81653c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:25.181366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 184/382] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:25.181987 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:25.637921 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae2313-9c86-44a1-961e-7cbc11fba603 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:25.638439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae2313-9c86-44a1-961e-7cbc11fba603 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:25.638439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae2313-9c86-44a1-961e-7cbc11fba603 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:25.640377 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 199/383] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:25.645528 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc2810-1980-4698-8034-2eb65bfd4b88 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:25.646076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc2810-1980-4698-8034-2eb65bfd4b88 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:25.646561 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc2810-1980-4698-8034-2eb65bfd4b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:25.678063 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-f6cc2810-1980-4698-8034-2eb65bfd4b88 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:25.682459 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f6cc2810-1980-4698-8034-2eb65bfd4b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:25.688610 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae2313-9c86-44a1-961e-7cbc11fba603 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:25.812910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 185/384] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:25.852608 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae2313-9c86-44a1-961e-7cbc11fba603 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:25.897388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3197b3-9c1a-4637-a721-09000b3f2eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:25.897946 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3197b3-9c1a-4637-a721-09000b3f2eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:25.898224 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3197b3-9c1a-4637-a721-09000b3f2eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:25.898497 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3197b3-9c1a-4637-a721-09000b3f2eb1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:25.912708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3197b3-9c1a-4637-a721-09000b3f2eb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:25.918136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-0b3197b3-9c1a-4637-a721-09000b3f2eb1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:59:25.918136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:25.918136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:25.918136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:25.918136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:25.918136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:25.918136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:25.918136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:25.918136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:25.918136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:25.918136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:25.918136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:25.918136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:25.918136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:25.918136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:25.918136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:25.919019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:25.919019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:25.919019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:25.919019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:25.919019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:25.919019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:25.919019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:25.919019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:25.919019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:25.919019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:25.919019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:25.919019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:25.919019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:25.919019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:25.919019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:25.919019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:25.919897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:25.919897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:25.919897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:25.919897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:25.919897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:25.919897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:25.919897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:25.919897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:25.919897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:25.919897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:25.919897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:59:25.919897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:25.922862 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 200/385] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 06:59:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:25.930194 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc2810-1980-4698-8034-2eb65bfd4b88 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:26.075512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc2810-1980-4698-8034-2eb65bfd4b88 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:26.110441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:26.118061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe5219-1844-47ab-b7c0-51a22a0ff6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:26.118883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe5219-1844-47ab-b7c0-51a22a0ff6ee None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:26.119226 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe5219-1844-47ab-b7c0-51a22a0ff6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:26.119686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe5219-1844-47ab-b7c0-51a22a0ff6ee None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:26.120587 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe5219-1844-47ab-b7c0-51a22a0ff6ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:26.120884 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe5219-1844-47ab-b7c0-51a22a0ff6ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:26.122294 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe5219-1844-47ab-b7c0-51a22a0ff6ee None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:26.134943 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 186/386] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:26.140576 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3197b3-9c1a-4637-a721-09000b3f2eb1 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:26.143620 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:26.179919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe5219-1844-47ab-b7c0-51a22a0ff6ee None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:26.313258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3197b3-9c1a-4637-a721-09000b3f2eb1 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:26.359164 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe5219-1844-47ab-b7c0-51a22a0ff6ee None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:26.374889 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b2414-a2b6-4e36-8c2e-af564e58ae2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:26.377382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b2414-a2b6-4e36-8c2e-af564e58ae2c None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:26.377904 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b2414-a2b6-4e36-8c2e-af564e58ae2c None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:26.378380 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b2414-a2b6-4e36-8c2e-af564e58ae2c None admin] PATH_INFO: `/v3/groups/8722ee6835904279adfb53a5649018b7` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:26.380462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b2414-a2b6-4e36-8c2e-af564e58ae2c None admin] RBAC: Authorizing `identity:get_group(group_id=8722ee6835904279adfb53a5649018b7)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:26.384159 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b2414-a2b6-4e36-8c2e-af564e58ae2c None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:26.385963 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 201/387] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 23 06:59:26 2022] GET /identity/v3/groups/8722ee6835904279adfb53a5649018b7 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:26.386964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e05ed-9678-4544-906f-ead6db97509b None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:26.387249 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e05ed-9678-4544-906f-ead6db97509b None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:26.387818 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e05ed-9678-4544-906f-ead6db97509b None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:26.389096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 202/388] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:26.394704 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e05ed-9678-4544-906f-ead6db97509b None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:26.424336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d391c30-3f83-45dd-9a29-e98e9faac6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:26.425410 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d391c30-3f83-45dd-9a29-e98e9faac6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:26.425894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d391c30-3f83-45dd-9a29-e98e9faac6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:26.426390 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d391c30-3f83-45dd-9a29-e98e9faac6a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:26.438284 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d391c30-3f83-45dd-9a29-e98e9faac6a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:26.448004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-9d391c30-3f83-45dd-9a29-e98e9faac6a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:26.448004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:26.448004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:26.448004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:26.448004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:26.448004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:26.448004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:26.448004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:26.448004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:26.448004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:26.448004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:26.448004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:26.448004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:26.448004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:26.448004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:26.448004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:26.448994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:26.448994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:26.448994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:26.448994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:26.448994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:26.448994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:26.448994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:26.448994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:26.448994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:26.448994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:26.448994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:26.448994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:26.448994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:26.448994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:26.448994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:26.448994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:26.449843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:26.449843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:26.449843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:26.449843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:26.449843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:26.449843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:26.449843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:26.449843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:26.449843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:26.449843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:26.449843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:26.449843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:26.452246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 187/389] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:26.453792 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d803ad5-b091-4a4d-9a64-7267cf277dc1 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:26.454344 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d803ad5-b091-4a4d-9a64-7267cf277dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:26.454817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d803ad5-b091-4a4d-9a64-7267cf277dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:26.480922 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-2d803ad5-b091-4a4d-9a64-7267cf277dc1 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:26.482165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2d803ad5-b091-4a4d-9a64-7267cf277dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:26.507784 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:26.548802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:26.550878 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e05ed-9678-4544-906f-ead6db97509b None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:26.579838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 188/390] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:26.580726 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d803ad5-b091-4a4d-9a64-7267cf277dc1 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:26.600471 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d898d7-7463-4c5c-891b-c87d126bb677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:26.601760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d898d7-7463-4c5c-891b-c87d126bb677 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:26.602245 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d898d7-7463-4c5c-891b-c87d126bb677 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:26.602727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d898d7-7463-4c5c-891b-c87d126bb677 None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:26.604186 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d898d7-7463-4c5c-891b-c87d126bb677 None admin] RBAC: Authorizing `identity:get_project(project_id=92ca046a717e4adab11f7371ad3cc888)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:26.606682 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d898d7-7463-4c5c-891b-c87d126bb677 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:26.609905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 203/391] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:26 2022] GET /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:26.616469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d898d7-7463-4c5c-891b-c87d126bb677 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:26.716258 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d803ad5-b091-4a4d-9a64-7267cf277dc1 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:26.760490 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9dc8a6-57df-4696-9dcc-d7e49bd9009c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:26.761644 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9dc8a6-57df-4696-9dcc-d7e49bd9009c None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:26.762424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9dc8a6-57df-4696-9dcc-d7e49bd9009c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:26.763214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9dc8a6-57df-4696-9dcc-d7e49bd9009c None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:26.764113 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9dc8a6-57df-4696-9dcc-d7e49bd9009c None admin] RBAC: Adding query filter params (name=member) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:26.764642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9dc8a6-57df-4696-9dcc-d7e49bd9009c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:26.764812 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d898d7-7463-4c5c-891b-c87d126bb677 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:26.766949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9dc8a6-57df-4696-9dcc-d7e49bd9009c None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:26.787867 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 189/392] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:59:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:26.794240 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9dc8a6-57df-4696-9dcc-d7e49bd9009c None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:26.828093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34855b7c-c8cd-412e-8782-e95aaebab15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:26.829173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34855b7c-c8cd-412e-8782-e95aaebab15e None admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:26.829445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34855b7c-c8cd-412e-8782-e95aaebab15e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:26.829715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34855b7c-c8cd-412e-8782-e95aaebab15e None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888/groups/8722ee6835904279adfb53a5649018b7/roles/c0ef5f54fa5e4cbb93cb48c9c02b83b7` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:26.835557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34855b7c-c8cd-412e-8782-e95aaebab15e None admin] RBAC: Authorizing `identity:create_grant(project_id=92ca046a717e4adab11f7371ad3cc888, group_id=8722ee6835904279adfb53a5649018b7, role_id=c0ef5f54fa5e4cbb93cb48c9c02b83b7)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:26.837204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34855b7c-c8cd-412e-8782-e95aaebab15e None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:26.857076 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:26.864368 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 204/393] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Aug 23 06:59:26 2022] PUT /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888/groups/8722ee6835904279adfb53a5649018b7/roles/c0ef5f54fa5e4cbb93cb48c9c02b83b7 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:59:26.873353 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:26.946525 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9dc8a6-57df-4696-9dcc-d7e49bd9009c None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:26.956196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34855b7c-c8cd-412e-8782-e95aaebab15e None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:27.023345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c1949-7e8b-4f84-9971-1b097e446384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:27.024430 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c1949-7e8b-4f84-9971-1b097e446384 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:27.024918 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c1949-7e8b-4f84-9971-1b097e446384 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:27.025400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c1949-7e8b-4f84-9971-1b097e446384 None admin] PATH_INFO: `/v3/groups/8722ee6835904279adfb53a5649018b7` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:27.032391 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c1949-7e8b-4f84-9971-1b097e446384 None admin] RBAC: Authorizing `identity:get_group(group_id=8722ee6835904279adfb53a5649018b7)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:27.034307 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c1949-7e8b-4f84-9971-1b097e446384 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:27.036806 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 190/394] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 23 06:59:26 2022] GET /identity/v3/groups/8722ee6835904279adfb53a5649018b7 => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:27.041891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c1949-7e8b-4f84-9971-1b097e446384 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:27.095256 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34855b7c-c8cd-412e-8782-e95aaebab15e None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:27.132585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3310a9a-334a-4ee4-b7e1-6e2b26b36de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:27.133217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3310a9a-334a-4ee4-b7e1-6e2b26b36de2 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:27.133489 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3310a9a-334a-4ee4-b7e1-6e2b26b36de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:27.133754 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3310a9a-334a-4ee4-b7e1-6e2b26b36de2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:27.147545 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3310a9a-334a-4ee4-b7e1-6e2b26b36de2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:27.154050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-b3310a9a-334a-4ee4-b7e1-6e2b26b36de2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:27.154050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:27.154050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:27.154050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:27.154050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:27.154050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:27.154050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:27.154050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:27.154050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:27.154050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:27.154050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:27.154050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:27.154050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:27.154050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:27.154050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:27.154050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:27.154953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:27.154953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:27.154953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:27.154953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:27.154953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:27.154953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:27.154953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:27.154953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:27.154953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:27.154953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:27.154953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:27.154953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:27.154953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:27.154953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:27.154953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:27.154953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:27.155857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:27.155857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:27.155857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:27.155857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:27.155857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:27.155857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:27.155857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:27.155857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:27.155857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:27.155857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:27.155857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:27.155857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:27.156460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 205/395] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:27.159447 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d123bd3-5c58-4c7e-8d5f-59395296435e None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:27.159447 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d123bd3-5c58-4c7e-8d5f-59395296435e None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:27.159447 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d123bd3-5c58-4c7e-8d5f-59395296435e None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:27.160429 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 206/396] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:27.164341 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d123bd3-5c58-4c7e-8d5f-59395296435e None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:27.168779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c1949-7e8b-4f84-9971-1b097e446384 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:27.182935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:27.213384 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5541c6-ce67-4df6-971a-23094915cd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:27.214165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5541c6-ce67-4df6-971a-23094915cd7e None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:27.214511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5541c6-ce67-4df6-971a-23094915cd7e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:27.214852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5541c6-ce67-4df6-971a-23094915cd7e None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:27.216281 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5541c6-ce67-4df6-971a-23094915cd7e None admin] RBAC: Authorizing `identity:get_project(project_id=92ca046a717e4adab11f7371ad3cc888)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:27.218509 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5541c6-ce67-4df6-971a-23094915cd7e None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:27.221432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 191/397] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:27 2022] GET /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:27.222714 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1334ce9-2609-4502-b5d9-8a6a1674d494 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:27.223067 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1334ce9-2609-4502-b5d9-8a6a1674d494 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:27.223431 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1334ce9-2609-4502-b5d9-8a6a1674d494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:27.245253 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:27.247102 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-f1334ce9-2609-4502-b5d9-8a6a1674d494 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:27.248717 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f1334ce9-2609-4502-b5d9-8a6a1674d494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:27.292871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d123bd3-5c58-4c7e-8d5f-59395296435e None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:27.328510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 192/398] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:27.332447 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1334ce9-2609-4502-b5d9-8a6a1674d494 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:27.336082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bb4ca-7fc4-4dca-a0b2-32f27a67129e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:27.336699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bb4ca-7fc4-4dca-a0b2-32f27a67129e None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:27.336969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bb4ca-7fc4-4dca-a0b2-32f27a67129e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:27.337236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bb4ca-7fc4-4dca-a0b2-32f27a67129e None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:27.337902 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918bb4ca-7fc4-4dca-a0b2-32f27a67129e None admin] RBAC: Adding query filter params (name=member) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:27.338199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918bb4ca-7fc4-4dca-a0b2-32f27a67129e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:27.339608 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918bb4ca-7fc4-4dca-a0b2-32f27a67129e None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:27.359705 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 207/399] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:59:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:27.365441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bb4ca-7fc4-4dca-a0b2-32f27a67129e None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:27.477268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bb4ca-7fc4-4dca-a0b2-32f27a67129e None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:27.503105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:27.513449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f864ba6-06ba-482f-bbdc-cb71ab52c41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:27.514530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f864ba6-06ba-482f-bbdc-cb71ab52c41c None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:27.514900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f864ba6-06ba-482f-bbdc-cb71ab52c41c None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:27.515316 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f864ba6-06ba-482f-bbdc-cb71ab52c41c None admin] PATH_INFO: `/v3/groups/8722ee6835904279adfb53a5649018b7` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:27.516551 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f864ba6-06ba-482f-bbdc-cb71ab52c41c None admin] RBAC: Authorizing `identity:get_group(group_id=8722ee6835904279adfb53a5649018b7)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:27.518101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f864ba6-06ba-482f-bbdc-cb71ab52c41c None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:27.520368 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 208/400] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 23 06:59:27 2022] GET /identity/v3/groups/8722ee6835904279adfb53a5649018b7 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:27.532608 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f864ba6-06ba-482f-bbdc-cb71ab52c41c None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:27.537815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1334ce9-2609-4502-b5d9-8a6a1674d494 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:27.548236 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:27.578196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc85077-150c-4c6f-9b43-403e9795de1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:27.578923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc85077-150c-4c6f-9b43-403e9795de1c None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:27.579259 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc85077-150c-4c6f-9b43-403e9795de1c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:27.579700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc85077-150c-4c6f-9b43-403e9795de1c None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888/groups/8722ee6835904279adfb53a5649018b7/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:27.581732 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc85077-150c-4c6f-9b43-403e9795de1c None admin] RBAC: Authorizing `identity:create_grant(project_id=92ca046a717e4adab11f7371ad3cc888, group_id=8722ee6835904279adfb53a5649018b7, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:27.583217 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc85077-150c-4c6f-9b43-403e9795de1c None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:27.599842 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 193/401] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Aug 23 06:59:27 2022] PUT /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888/groups/8722ee6835904279adfb53a5649018b7/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:59:27.608010 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6803f7f-c828-4bb5-9e0d-4e0d01770944 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:27.608345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6803f7f-c828-4bb5-9e0d-4e0d01770944 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:27.608686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6803f7f-c828-4bb5-9e0d-4e0d01770944 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:27.610719 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 194/402] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:27.616124 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7f3f0-e7b6-46d3-af33-5d1bc61182c3 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:27.616464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7f3f0-e7b6-46d3-af33-5d1bc61182c3 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:27.616745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7f3f0-e7b6-46d3-af33-5d1bc61182c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:27.642309 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f864ba6-06ba-482f-bbdc-cb71ab52c41c None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:27.648568 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-55c7f3f0-e7b6-46d3-af33-5d1bc61182c3 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:27.649807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-55c7f3f0-e7b6-46d3-af33-5d1bc61182c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:27.695146 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfefe7-1835-4fe8-bcae-c777654a3def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:27.695812 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfefe7-1835-4fe8-bcae-c777654a3def None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:27.696088 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfefe7-1835-4fe8-bcae-c777654a3def None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:27.696357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfefe7-1835-4fe8-bcae-c777654a3def None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:27.697817 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfefe7-1835-4fe8-bcae-c777654a3def None admin] RBAC: Authorizing `identity:get_project(project_id=e594e988024e47b4a48b43d8f828c605)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:27.699281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfefe7-1835-4fe8-bcae-c777654a3def None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:27.701304 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 209/403] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:27 2022] GET /identity/v3/projects/e594e988024e47b4a48b43d8f828c605 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:27.702225 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfefe7-1835-4fe8-bcae-c777654a3def None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:27.765620 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 195/404] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:59:27.767451 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:27.768302 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7f3f0-e7b6-46d3-af33-5d1bc61182c3 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:27.819458 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfefe7-1835-4fe8-bcae-c777654a3def None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:27.854858 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0ef4a-c570-4833-b34c-27084313e97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:27.855504 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0ef4a-c570-4833-b34c-27084313e97c None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:27.855782 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0ef4a-c570-4833-b34c-27084313e97c None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:27.856048 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0ef4a-c570-4833-b34c-27084313e97c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:27.866936 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0ef4a-c570-4833-b34c-27084313e97c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:27.871676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-3ac0ef4a-c570-4833-b34c-27084313e97c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:59:27.871676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:27.871676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:27.871676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:27.871676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:27.871676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:27.871676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:27.871676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:27.871676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:27.871676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:27.871676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:27.871676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:27.871676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:27.871676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:27.871676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:27.871676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:27.872534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:27.872534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:27.872534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:27.872534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:27.872534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:27.872534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:27.872534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:27.872534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:27.872534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:27.872534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:27.872534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:27.872534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:27.872534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:27.872534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:27.872534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:27.872534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:27.873405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:27.873405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:27.873405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:27.873405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:27.873405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:27.873405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:27.873405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:27.873405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:27.873405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:27.873405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:27.873405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:59:27.873405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:27.873405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 210/405] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 06:59:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:27.877970 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:27.880635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0ef4a-c570-4833-b34c-27084313e97c None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:27.882668 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7f3f0-e7b6-46d3-af33-5d1bc61182c3 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:27.927234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c12c7-fa76-4316-aa5d-bc62110463bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:27.928223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c12c7-fa76-4316-aa5d-bc62110463bd None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:27.929170 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c12c7-fa76-4316-aa5d-bc62110463bd None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:27.929671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c12c7-fa76-4316-aa5d-bc62110463bd None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605/groups/8722ee6835904279adfb53a5649018b7/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:27.931857 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c12c7-fa76-4316-aa5d-bc62110463bd None admin] RBAC: Authorizing `identity:create_grant(project_id=e594e988024e47b4a48b43d8f828c605, group_id=8722ee6835904279adfb53a5649018b7, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:27.934042 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c12c7-fa76-4316-aa5d-bc62110463bd None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:27.954255 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 196/406] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Aug 23 06:59:27 2022] PUT /identity/v3/projects/e594e988024e47b4a48b43d8f828c605/groups/8722ee6835904279adfb53a5649018b7/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:59:28.019469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0ef4a-c570-4833-b34c-27084313e97c None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:28.040475 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:28.061557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b9af3-8b01-49f6-9636-347c64b0c249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:28.062618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b9af3-8b01-49f6-9636-347c64b0c249 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:28.063079 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b9af3-8b01-49f6-9636-347c64b0c249 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:28.063542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b9af3-8b01-49f6-9636-347c64b0c249 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:28.064410 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b9af3-8b01-49f6-9636-347c64b0c249 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:28.064938 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b9af3-8b01-49f6-9636-347c64b0c249 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:28.066572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b9af3-8b01-49f6-9636-347c64b0c249 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:28.077222 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 211/407] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:28.084221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c12c7-fa76-4316-aa5d-bc62110463bd None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:28.100115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b9af3-8b01-49f6-9636-347c64b0c249 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:28.210392 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c12c7-fa76-4316-aa5d-bc62110463bd None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:28.212129 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b9af3-8b01-49f6-9636-347c64b0c249 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:28.248721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d7021-95d9-4393-a048-d13e17ca4551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:28.249672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d7021-95d9-4393-a048-d13e17ca4551 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:28.250126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d7021-95d9-4393-a048-d13e17ca4551 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:28.250562 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d7021-95d9-4393-a048-d13e17ca4551 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:28.261712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce29a5-7e9a-4cd6-a302-1f50621d8107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:28.262368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce29a5-7e9a-4cd6-a302-1f50621d8107 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:28.262724 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce29a5-7e9a-4cd6-a302-1f50621d8107 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:28.262907 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce29a5-7e9a-4cd6-a302-1f50621d8107 None admin] PATH_INFO: `/v3/groups/8722ee6835904279adfb53a5649018b7` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:28.263932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d7021-95d9-4393-a048-d13e17ca4551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:28.264100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce29a5-7e9a-4cd6-a302-1f50621d8107 None admin] RBAC: Authorizing `identity:get_group(group_id=8722ee6835904279adfb53a5649018b7)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:28.265615 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce29a5-7e9a-4cd6-a302-1f50621d8107 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:28.269867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-4a1d7021-95d9-4393-a048-d13e17ca4551 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:59:28.269867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:28.269867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:28.269867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:28.269867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:28.269867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:28.269867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:28.269867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:28.269867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:28.269867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:28.269867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:28.269867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:28.269867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:28.269867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:28.269867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:28.269867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:28.270948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:28.270948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:28.270948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:28.270948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:28.270948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:28.270948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:28.270948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:28.270948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:28.270948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:28.270948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:28.270948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:28.270948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:28.270948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:28.270948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:28.270948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:28.270948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:28.271993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:28.271993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:28.271993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:28.271993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:28.271993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:28.271993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:28.271993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:28.271993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:28.271993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:28.271993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:28.271993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:59:28.271993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:28.274263 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 212/408] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 06:59:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:28.274648 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 197/409] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 23 06:59:28 2022] GET /identity/v3/groups/8722ee6835904279adfb53a5649018b7 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:28.279942 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d7021-95d9-4393-a048-d13e17ca4551 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:28.280083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce29a5-7e9a-4cd6-a302-1f50621d8107 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:28.323250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:28.375894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:28.388401 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d7021-95d9-4393-a048-d13e17ca4551 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:28.424477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f43e4-4f4e-47a3-8d0c-85a1da767357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:28.425333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f43e4-4f4e-47a3-8d0c-85a1da767357 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:28.425936 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f43e4-4f4e-47a3-8d0c-85a1da767357 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:28.426481 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f43e4-4f4e-47a3-8d0c-85a1da767357 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:28.427309 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f43e4-4f4e-47a3-8d0c-85a1da767357 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:28.427788 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f43e4-4f4e-47a3-8d0c-85a1da767357 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:28.429323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f43e4-4f4e-47a3-8d0c-85a1da767357 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:28.439040 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 213/410] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:28.444397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f43e4-4f4e-47a3-8d0c-85a1da767357 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:28.484542 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce29a5-7e9a-4cd6-a302-1f50621d8107 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:28.552978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df0a8c-8335-4262-b263-dee8fec57bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:28.553626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df0a8c-8335-4262-b263-dee8fec57bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:28.553899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df0a8c-8335-4262-b263-dee8fec57bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:28.554168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df0a8c-8335-4262-b263-dee8fec57bb1 None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:28.559967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df0a8c-8335-4262-b263-dee8fec57bb1 None admin] RBAC: Authorizing `identity:get_project(project_id=e594e988024e47b4a48b43d8f828c605)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:28.561426 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df0a8c-8335-4262-b263-dee8fec57bb1 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:28.565587 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 198/411] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:28 2022] GET /identity/v3/projects/e594e988024e47b4a48b43d8f828c605 => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:28.570015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df0a8c-8335-4262-b263-dee8fec57bb1 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:28.613061 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f43e4-4f4e-47a3-8d0c-85a1da767357 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:28.623733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:28.681335 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123026f-da81-4221-b0ca-af361f68197a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:28.681969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123026f-da81-4221-b0ca-af361f68197a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:28.682245 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123026f-da81-4221-b0ca-af361f68197a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:28.682514 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123026f-da81-4221-b0ca-af361f68197a None admin] PATH_INFO: `/v3/groups/3a712ed666f84aecbb18f75796c91ad3` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:28.684581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123026f-da81-4221-b0ca-af361f68197a None admin] RBAC: Authorizing `identity:get_group(group_id=3a712ed666f84aecbb18f75796c91ad3)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:28.686034 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123026f-da81-4221-b0ca-af361f68197a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:28.692416 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 214/412] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 23 06:59:28 2022] GET /identity/v3/groups/3a712ed666f84aecbb18f75796c91ad3 => generated 229 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:28.695503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123026f-da81-4221-b0ca-af361f68197a None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:28.748206 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:28.756876 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df0a8c-8335-4262-b263-dee8fec57bb1 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:28.824133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35389b5-e329-4037-a973-758e9e10abd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:28.824797 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35389b5-e329-4037-a973-758e9e10abd1 None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:28.825069 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35389b5-e329-4037-a973-758e9e10abd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:28.825339 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35389b5-e329-4037-a973-758e9e10abd1 None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605/groups/8722ee6835904279adfb53a5649018b7/roles/c0ef5f54fa5e4cbb93cb48c9c02b83b7` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:28.832733 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35389b5-e329-4037-a973-758e9e10abd1 None admin] RBAC: Authorizing `identity:create_grant(project_id=e594e988024e47b4a48b43d8f828c605, group_id=8722ee6835904279adfb53a5649018b7, role_id=c0ef5f54fa5e4cbb93cb48c9c02b83b7)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:28.834190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35389b5-e329-4037-a973-758e9e10abd1 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:28.855809 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 199/413] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Aug 23 06:59:28 2022] PUT /identity/v3/projects/e594e988024e47b4a48b43d8f828c605/groups/8722ee6835904279adfb53a5649018b7/roles/c0ef5f54fa5e4cbb93cb48c9c02b83b7 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 06:59:28.871536 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123026f-da81-4221-b0ca-af361f68197a None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:28.917941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81be8a9-d179-4c50-8e26-7756d3f3bf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:28.918594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81be8a9-d179-4c50-8e26-7756d3f3bf03 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:28.918869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81be8a9-d179-4c50-8e26-7756d3f3bf03 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:28.919189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81be8a9-d179-4c50-8e26-7756d3f3bf03 None admin] PATH_INFO: `/v3/projects/c7014afbcdaa44f68b1b75e344808f41` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:28.920467 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81be8a9-d179-4c50-8e26-7756d3f3bf03 None admin] RBAC: Authorizing `identity:get_project(project_id=c7014afbcdaa44f68b1b75e344808f41)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:28.922100 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81be8a9-d179-4c50-8e26-7756d3f3bf03 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:28.927630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 215/414] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:28 2022] GET /identity/v3/projects/c7014afbcdaa44f68b1b75e344808f41 => generated 344 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:28.930916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35389b5-e329-4037-a973-758e9e10abd1 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:29.067141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35389b5-e329-4037-a973-758e9e10abd1 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:29.111451 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996015be-a60a-4208-9dd1-ca95e514974f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:29.112175 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996015be-a60a-4208-9dd1-ca95e514974f None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:29.112458 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996015be-a60a-4208-9dd1-ca95e514974f None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:29.112770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996015be-a60a-4208-9dd1-ca95e514974f None admin] PATH_INFO: `/v3/projects/c7014afbcdaa44f68b1b75e344808f41/groups/3a712ed666f84aecbb18f75796c91ad3/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:29.114690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996015be-a60a-4208-9dd1-ca95e514974f None admin] RBAC: Authorizing `identity:create_grant(project_id=c7014afbcdaa44f68b1b75e344808f41, group_id=3a712ed666f84aecbb18f75796c91ad3, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:29.116214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996015be-a60a-4208-9dd1-ca95e514974f None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:29.132550 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 200/415] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Aug 23 06:59:28 2022] PUT /identity/v3/projects/c7014afbcdaa44f68b1b75e344808f41/groups/3a712ed666f84aecbb18f75796c91ad3/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:59:29.241447 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72815acc-7070-499a-bd8a-755426c93b54 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:29.241447 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72815acc-7070-499a-bd8a-755426c93b54 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:29.241447 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72815acc-7070-499a-bd8a-755426c93b54 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:29.242141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 216/416] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:29.248150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b453e1-7895-4946-b59f-75b2277e7c98 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:29.249098 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b453e1-7895-4946-b59f-75b2277e7c98 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:29.249571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b453e1-7895-4946-b59f-75b2277e7c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:29.276552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-13b453e1-7895-4946-b59f-75b2277e7c98 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:29.277850 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-13b453e1-7895-4946-b59f-75b2277e7c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:29.289616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:29.360582 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 201/417] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:29.697378 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72815acc-7070-499a-bd8a-755426c93b54 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:29.743764 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:29.816796 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72815acc-7070-499a-bd8a-755426c93b54 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:29.851756 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97823fb-2bfa-436d-8aea-679b1beaee38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:29.852393 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97823fb-2bfa-436d-8aea-679b1beaee38 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:29.852674 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97823fb-2bfa-436d-8aea-679b1beaee38 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:29.852949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97823fb-2bfa-436d-8aea-679b1beaee38 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:29.863492 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97823fb-2bfa-436d-8aea-679b1beaee38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:29.869089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-c97823fb-2bfa-436d-8aea-679b1beaee38 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:59:29.869089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:29.869089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:29.869089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:29.869089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:29.869089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:29.869089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:29.869089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:29.869089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:29.869089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:29.869089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:29.869089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:29.869089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:29.869089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:29.869089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:29.869089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:29.870178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:29.870178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:29.870178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:29.870178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:29.870178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:29.870178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:29.870178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:29.870178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:29.870178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:29.870178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:29.870178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:29.870178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:29.870178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:29.870178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:29.870178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:29.870178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:29.871183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:29.871183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:29.871183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:29.871183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:29.871183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:29.871183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:29.871183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:29.871183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:29.871183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:29.871183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:29.871183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:59:29.871183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:29.873897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 217/418] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 06:59:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:29.879610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b453e1-7895-4946-b59f-75b2277e7c98 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:29.984612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eeb4b8-2b16-4a06-aa34-3d27affdfece None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:29.985320 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eeb4b8-2b16-4a06-aa34-3d27affdfece None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:29.985825 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eeb4b8-2b16-4a06-aa34-3d27affdfece None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:29.991735 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 218/419] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:59:29.996782 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d87783-0112-4647-a84d-83ad9cdd7437 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:29.997314 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d87783-0112-4647-a84d-83ad9cdd7437 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:29.997787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d87783-0112-4647-a84d-83ad9cdd7437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:30.011452 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b453e1-7895-4946-b59f-75b2277e7c98 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:30.032607 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-45d87783-0112-4647-a84d-83ad9cdd7437 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:30.034022 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-45d87783-0112-4647-a84d-83ad9cdd7437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:30.065965 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ae8f0-7d4e-4b67-9edc-ce3f42761033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:30.066911 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ae8f0-7d4e-4b67-9edc-ce3f42761033 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:30.067320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ae8f0-7d4e-4b67-9edc-ce3f42761033 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:30.068670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ae8f0-7d4e-4b67-9edc-ce3f42761033 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:30.069484 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ae8f0-7d4e-4b67-9edc-ce3f42761033 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:30.069928 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ae8f0-7d4e-4b67-9edc-ce3f42761033 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:30.071548 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ae8f0-7d4e-4b67-9edc-ce3f42761033 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:30.082128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 202/420] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:30.089879 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ae8f0-7d4e-4b67-9edc-ce3f42761033 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:30.096913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:30.185724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 219/421] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:30.205365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ae8f0-7d4e-4b67-9edc-ce3f42761033 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:30.241340 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aabe73-b4e3-425f-8ad6-6157e564b066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:30.242179 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aabe73-b4e3-425f-8ad6-6157e564b066 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:30.242613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aabe73-b4e3-425f-8ad6-6157e564b066 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:30.243025 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aabe73-b4e3-425f-8ad6-6157e564b066 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:30.316398 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aabe73-b4e3-425f-8ad6-6157e564b066 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:30.318658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aabe73-b4e3-425f-8ad6-6157e564b066 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:30.328764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:30.428527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 203/422] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 06:59:30.433938 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d87783-0112-4647-a84d-83ad9cdd7437 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:30.462899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4ecb4-1020-4e59-a434-5a70a4dceec9 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:30.463231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4ecb4-1020-4e59-a434-5a70a4dceec9 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:30.463475 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4ecb4-1020-4e59-a434-5a70a4dceec9 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:30.465958 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 204/423] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:59:30.470888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afdd84-33bd-40b1-854e-b025080f9d17 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:30.471166 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afdd84-33bd-40b1-854e-b025080f9d17 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:30.471468 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afdd84-33bd-40b1-854e-b025080f9d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:30.489772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:30.491309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-36afdd84-33bd-40b1-854e-b025080f9d17 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:30.492629 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-36afdd84-33bd-40b1-854e-b025080f9d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:30.571151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 205/424] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:59:30.576271 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afdd84-33bd-40b1-854e-b025080f9d17 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:30.624860 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:30.665733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d87783-0112-4647-a84d-83ad9cdd7437 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:30.693083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afdd84-33bd-40b1-854e-b025080f9d17 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:30.703578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2298a0-8e23-4775-bdbe-c5205cb228f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:30.704234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2298a0-8e23-4775-bdbe-c5205cb228f8 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:30.704507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2298a0-8e23-4775-bdbe-c5205cb228f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:30.704772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2298a0-8e23-4775-bdbe-c5205cb228f8 None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:30.712166 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2298a0-8e23-4775-bdbe-c5205cb228f8 None admin] RBAC: Authorizing `identity:get_project(project_id=92ca046a717e4adab11f7371ad3cc888)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:30.713630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2298a0-8e23-4775-bdbe-c5205cb228f8 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:30.715265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 220/425] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:30 2022] GET /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888 => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:30.721241 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2298a0-8e23-4775-bdbe-c5205cb228f8 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:30.773981 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffe3f0-7935-4860-ad9d-914f392611fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:30.774609 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffe3f0-7935-4860-ad9d-914f392611fe None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:30.774885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffe3f0-7935-4860-ad9d-914f392611fe None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:30.775153 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffe3f0-7935-4860-ad9d-914f392611fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:30.812053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cffe3f0-7935-4860-ad9d-914f392611fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:30.817119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-8cffe3f0-7935-4860-ad9d-914f392611fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:30.817119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:30.817119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:30.817119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:30.817119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:30.817119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:30.817119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:30.817119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:30.817119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:30.817119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:30.817119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:30.817119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:30.817119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:30.817119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:30.817119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:30.817119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:30.818000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:30.818000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:30.818000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:30.818000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:30.818000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:30.818000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:30.818000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:30.818000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:30.818000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:30.818000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:30.818000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:30.818000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:30.818000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:30.818000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:30.818000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:30.818000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:30.818872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:30.818872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:30.818872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:30.818872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:30.818872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:30.818872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:30.818872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:30.818872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:30.818872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:30.818872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:30.818872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:30.818872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:30.818872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 206/426] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:30.824296 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffe3f0-7935-4860-ad9d-914f392611fe None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:30.864681 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:30.873050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2298a0-8e23-4775-bdbe-c5205cb228f8 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:30.910781 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4f0ad-5315-447f-8d2e-97b67924d1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:30.911965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4f0ad-5315-447f-8d2e-97b67924d1c8 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:30.912378 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4f0ad-5315-447f-8d2e-97b67924d1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:30.912813 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4f0ad-5315-447f-8d2e-97b67924d1c8 None admin] PATH_INFO: `/v3/groups/8722ee6835904279adfb53a5649018b7` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:30.914060 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4f0ad-5315-447f-8d2e-97b67924d1c8 None admin] RBAC: Authorizing `identity:get_group(group_id=8722ee6835904279adfb53a5649018b7)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:30.915985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4f0ad-5315-447f-8d2e-97b67924d1c8 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:30.918084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 221/427] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 23 06:59:30 2022] GET /identity/v3/groups/8722ee6835904279adfb53a5649018b7 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:30.923789 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4f0ad-5315-447f-8d2e-97b67924d1c8 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:30.999116 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:31.007124 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffe3f0-7935-4860-ad9d-914f392611fe None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:31.048899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c52216-695e-4a1c-a4e7-8c1af52ea9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:31.050146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c52216-695e-4a1c-a4e7-8c1af52ea9fd None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:31.050767 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c52216-695e-4a1c-a4e7-8c1af52ea9fd None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:31.051203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c52216-695e-4a1c-a4e7-8c1af52ea9fd None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:31.052571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c52216-695e-4a1c-a4e7-8c1af52ea9fd None admin] RBAC: Adding query filter params (name=member) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:31.053260 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c52216-695e-4a1c-a4e7-8c1af52ea9fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:31.055187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4f0ad-5315-447f-8d2e-97b67924d1c8 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:31.055805 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c52216-695e-4a1c-a4e7-8c1af52ea9fd None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:31.065388 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 207/428] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:59:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:31.066593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c52216-695e-4a1c-a4e7-8c1af52ea9fd None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:31.107596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed82bf1-bb23-4ae6-b76c-a893009f3447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:31.108215 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed82bf1-bb23-4ae6-b76c-a893009f3447 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:31.108496 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed82bf1-bb23-4ae6-b76c-a893009f3447 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:31.108773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed82bf1-bb23-4ae6-b76c-a893009f3447 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:31.109545 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed82bf1-bb23-4ae6-b76c-a893009f3447 None admin] RBAC: Adding query filter params (group.id=8722ee6835904279adfb53a5649018b7, role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7, scope.project.id=92ca046a717e4adab11f7371ad3cc888) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:31.109858 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed82bf1-bb23-4ae6-b76c-a893009f3447 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:31.111312 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed82bf1-bb23-4ae6-b76c-a893009f3447 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:31.127496 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 222/429] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 23 06:59:30 2022] GET /identity/v3/role_assignments?group.id=8722ee6835904279adfb53a5649018b7&scope.project.id=92ca046a717e4adab11f7371ad3cc888&role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7 => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:31.127496 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed82bf1-bb23-4ae6-b76c-a893009f3447 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:31.155453 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:31.253429 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed82bf1-bb23-4ae6-b76c-a893009f3447 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:31.262431 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c52216-695e-4a1c-a4e7-8c1af52ea9fd None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:31.296554 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3152d67-9546-4343-a1c4-96e8ae5bc604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:31.297237 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3152d67-9546-4343-a1c4-96e8ae5bc604 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:31.297521 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3152d67-9546-4343-a1c4-96e8ae5bc604 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:31.297798 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3152d67-9546-4343-a1c4-96e8ae5bc604 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:31.313764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95dd1b-8375-4ada-b7a5-0e7cc09423be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:31.314325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95dd1b-8375-4ada-b7a5-0e7cc09423be None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:31.314599 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95dd1b-8375-4ada-b7a5-0e7cc09423be None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:31.314876 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95dd1b-8375-4ada-b7a5-0e7cc09423be None admin] PATH_INFO: `/v3/roles/member` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:31.327434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95dd1b-8375-4ada-b7a5-0e7cc09423be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:31.332909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-bf95dd1b-8375-4ada-b7a5-0e7cc09423be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:31.332909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:31.332909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:31.332909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:31.332909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:31.332909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:31.332909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:31.332909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:31.332909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:31.332909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:31.332909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:31.332909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:31.332909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:31.332909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:31.332909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:31.332909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:31.333874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:31.333874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:31.333874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:31.333874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:31.333874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:31.333874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:31.333874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:31.333874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:31.333874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:31.333874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:31.333874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:31.333874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:31.333874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:31.333874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:31.333874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:31.333874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:31.334686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:31.334686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:31.334686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:31.334686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:31.334686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:31.334686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:31.334686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:31.334686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:31.334686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:31.334686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:31.334686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 06:59:31.334686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:31.334686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 208/430] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:31.335363 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de99698-e114-4aa3-a7a1-112995183735 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:31.335967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de99698-e114-4aa3-a7a1-112995183735 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:31.335967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de99698-e114-4aa3-a7a1-112995183735 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:31.341490 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 209/431] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:31.344055 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd60d72-10b7-482c-8b82-ef7211749283 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:31.344055 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd60d72-10b7-482c-8b82-ef7211749283 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:31.344055 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd60d72-10b7-482c-8b82-ef7211749283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:31.369316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-3bd60d72-10b7-482c-8b82-ef7211749283 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:31.370528 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3bd60d72-10b7-482c-8b82-ef7211749283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:31.389786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:31.403812 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3152d67-9546-4343-a1c4-96e8ae5bc604 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:31.405445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3152d67-9546-4343-a1c4-96e8ae5bc604 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:31.437608 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:31.461070 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 210/432] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:59:31.464301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd60d72-10b7-482c-8b82-ef7211749283 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:31.526727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 223/433] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:31.527669 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3152d67-9546-4343-a1c4-96e8ae5bc604 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:31.596791 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd60d72-10b7-482c-8b82-ef7211749283 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:31.634356 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1389cd1-4634-4a8d-b8cc-063563996611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:31.634966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1389cd1-4634-4a8d-b8cc-063563996611 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:31.635244 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1389cd1-4634-4a8d-b8cc-063563996611 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:31.635550 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1389cd1-4634-4a8d-b8cc-063563996611 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:31.636015 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3152d67-9546-4343-a1c4-96e8ae5bc604 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:31.636221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1389cd1-4634-4a8d-b8cc-063563996611 None admin] RBAC: Adding query filter params (name=member) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:31.636528 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1389cd1-4634-4a8d-b8cc-063563996611 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:31.637941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1389cd1-4634-4a8d-b8cc-063563996611 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:31.646491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 211/434] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 06:59:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:31.647692 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f9306-a0c9-4edd-906d-a368f64299c7 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:31.647980 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f9306-a0c9-4edd-906d-a368f64299c7 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:31.648254 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f9306-a0c9-4edd-906d-a368f64299c7 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:31.649583 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 212/435] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:31.654179 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f9306-a0c9-4edd-906d-a368f64299c7 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:31.668335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:31.675041 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:31.687965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c986bb9-4cf6-492c-b529-ca5a187bb2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:31.688611 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c986bb9-4cf6-492c-b529-ca5a187bb2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:31.688942 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c986bb9-4cf6-492c-b529-ca5a187bb2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:31.689161 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c986bb9-4cf6-492c-b529-ca5a187bb2c0 None admin] PATH_INFO: `/v3/projects/92ca046a717e4adab11f7371ad3cc888` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:31.690308 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c986bb9-4cf6-492c-b529-ca5a187bb2c0 None admin] RBAC: Authorizing `identity:get_project(project_id=92ca046a717e4adab11f7371ad3cc888)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:31.691835 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c986bb9-4cf6-492c-b529-ca5a187bb2c0 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:31.693431 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 224/436] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:31 2022] GET /identity/v3/projects/92ca046a717e4adab11f7371ad3cc888 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:31.699267 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e18a63-81e5-4c62-b8c8-f6ec37046f61 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:31.699583 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e18a63-81e5-4c62-b8c8-f6ec37046f61 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:31.699948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e18a63-81e5-4c62-b8c8-f6ec37046f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:31.718501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-24e18a63-81e5-4c62-b8c8-f6ec37046f61 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:31.720295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-24e18a63-81e5-4c62-b8c8-f6ec37046f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:31.772469 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f9306-a0c9-4edd-906d-a368f64299c7 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:31.788424 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 225/437] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:31.789258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e18a63-81e5-4c62-b8c8-f6ec37046f61 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:31.805131 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e17cd-938b-490b-9061-02555d36b366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:31.805985 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e17cd-938b-490b-9061-02555d36b366 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:31.806432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e17cd-938b-490b-9061-02555d36b366 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:31.806843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e17cd-938b-490b-9061-02555d36b366 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:31.875373 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e17cd-938b-490b-9061-02555d36b366 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:31.876933 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e17cd-938b-490b-9061-02555d36b366 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:31.887824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:31.899067 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e18a63-81e5-4c62-b8c8-f6ec37046f61 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:31.925161 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:31.936819 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e47341-740c-4df5-be63-278d0184e4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:31.937448 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e47341-740c-4df5-be63-278d0184e4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:31.937829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e47341-740c-4df5-be63-278d0184e4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:31.938170 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e47341-740c-4df5-be63-278d0184e4f9 None admin] PATH_INFO: `/v3/groups/8722ee6835904279adfb53a5649018b7` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:31.939285 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e47341-740c-4df5-be63-278d0184e4f9 None admin] RBAC: Authorizing `identity:get_group(group_id=8722ee6835904279adfb53a5649018b7)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:31.940780 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e47341-740c-4df5-be63-278d0184e4f9 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:31.942740 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 226/438] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 23 06:59:31 2022] GET /identity/v3/groups/8722ee6835904279adfb53a5649018b7 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:31.943591 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e47341-740c-4df5-be63-278d0184e4f9 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:31.985497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 213/439] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:31.986358 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e17cd-938b-490b-9061-02555d36b366 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:32.060505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e47341-740c-4df5-be63-278d0184e4f9 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:32.091240 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e17cd-938b-490b-9061-02555d36b366 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:32.094865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a499f9f-eb5f-415c-b88c-19c88b7b5122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:32.095517 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a499f9f-eb5f-415c-b88c-19c88b7b5122 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:32.095797 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a499f9f-eb5f-415c-b88c-19c88b7b5122 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:32.096068 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a499f9f-eb5f-415c-b88c-19c88b7b5122 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:32.102040 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:32.106653 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a499f9f-eb5f-415c-b88c-19c88b7b5122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:32.111533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-4a499f9f-eb5f-415c-b88c-19c88b7b5122 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:59:32.111533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:32.111533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:32.111533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:32.111533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:32.111533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:32.111533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:32.111533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:32.111533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:32.111533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:32.111533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:32.111533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:32.111533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:32.111533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:32.111533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:32.111533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:32.112518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:32.112518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:32.112518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:32.112518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:32.112518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:32.112518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:32.112518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:32.112518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:32.112518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:32.112518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:32.112518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:32.112518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:32.112518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:32.112518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:32.112518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:32.112518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:32.113323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:32.113323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:32.113323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:32.113323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:32.113323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:32.113323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:32.113323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:32.113323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:32.113323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:32.113323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:32.113323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:59:32.113323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:32.113323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 227/440] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 06:59:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:32.113988 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a499f9f-eb5f-415c-b88c-19c88b7b5122 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:32.127367 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2a6dc-fc4b-4d15-b99d-4e848d72ad74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:32.127918 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2a6dc-fc4b-4d15-b99d-4e848d72ad74 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:32.128190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2a6dc-fc4b-4d15-b99d-4e848d72ad74 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:32.128447 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2a6dc-fc4b-4d15-b99d-4e848d72ad74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:32.129187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2a6dc-fc4b-4d15-b99d-4e848d72ad74 None admin] RBAC: Adding query filter params (group.id=8722ee6835904279adfb53a5649018b7, role.id=7670c7c450b9404ba57e12743cb6c458, scope.project.id=92ca046a717e4adab11f7371ad3cc888) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:32.129498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2a6dc-fc4b-4d15-b99d-4e848d72ad74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:32.130931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2a6dc-fc4b-4d15-b99d-4e848d72ad74 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:32.136626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 214/441] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 23 06:59:31 2022] GET /identity/v3/role_assignments?group.id=8722ee6835904279adfb53a5649018b7&scope.project.id=92ca046a717e4adab11f7371ad3cc888&role.id=7670c7c450b9404ba57e12743cb6c458 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:32.137391 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2a6dc-fc4b-4d15-b99d-4e848d72ad74 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:32.171058 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:32.236888 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a499f9f-eb5f-415c-b88c-19c88b7b5122 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:32.253989 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2a6dc-fc4b-4d15-b99d-4e848d72ad74 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:32.277731 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3da16-292f-4356-895d-a5e30f975a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:32.278373 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3da16-292f-4356-895d-a5e30f975a35 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:32.278647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3da16-292f-4356-895d-a5e30f975a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:32.278915 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3da16-292f-4356-895d-a5e30f975a35 None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:32.281218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3da16-292f-4356-895d-a5e30f975a35 None admin] RBAC: Authorizing `identity:get_project(project_id=e594e988024e47b4a48b43d8f828c605)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:32.282646 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3da16-292f-4356-895d-a5e30f975a35 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:32.285045 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 228/442] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:32 2022] GET /identity/v3/projects/e594e988024e47b4a48b43d8f828c605 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:32.285045 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3da16-292f-4356-895d-a5e30f975a35 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:32.299696 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6696da3-f0f2-4994-9615-f0947eb696ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:32.300651 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6696da3-f0f2-4994-9615-f0947eb696ee None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:32.301319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6696da3-f0f2-4994-9615-f0947eb696ee None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:32.301589 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6696da3-f0f2-4994-9615-f0947eb696ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:32.313783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6696da3-f0f2-4994-9615-f0947eb696ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:32.318923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-f6696da3-f0f2-4994-9615-f0947eb696ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:59:32.318923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:32.318923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:32.318923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:32.318923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:32.318923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:32.318923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:32.318923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:32.318923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:32.318923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:32.318923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:32.318923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:32.318923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:32.318923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:32.318923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:32.318923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:32.319888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:32.319888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:32.319888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:32.319888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:32.319888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:32.319888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:32.319888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:32.319888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:32.319888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:32.319888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:32.319888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:32.319888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:32.319888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:32.319888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:32.319888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:32.319888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:32.323306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:32.323306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:32.323306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:32.323306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:32.323306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:32.323306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:32.323306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:32.323306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:32.323306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:32.323306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:32.323306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:59:32.323306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:32.323306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 215/443] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 06:59:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:32.323306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6696da3-f0f2-4994-9615-f0947eb696ee None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:32.360963 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:32.410540 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3da16-292f-4356-895d-a5e30f975a35 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:32.432275 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:32.441883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6696da3-f0f2-4994-9615-f0947eb696ee None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:32.450782 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295c77d-51d7-426e-86c7-e9355709ed93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:32.451563 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295c77d-51d7-426e-86c7-e9355709ed93 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:32.451859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295c77d-51d7-426e-86c7-e9355709ed93 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:32.452124 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295c77d-51d7-426e-86c7-e9355709ed93 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:32.452802 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295c77d-51d7-426e-86c7-e9355709ed93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:32.453107 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295c77d-51d7-426e-86c7-e9355709ed93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:32.454594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295c77d-51d7-426e-86c7-e9355709ed93 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:32.464922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 229/444] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:32.465617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295c77d-51d7-426e-86c7-e9355709ed93 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:32.488026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433c341-0144-4a80-aaa1-73090afd457f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:32.488681 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433c341-0144-4a80-aaa1-73090afd457f None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:32.489003 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433c341-0144-4a80-aaa1-73090afd457f None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:32.489275 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433c341-0144-4a80-aaa1-73090afd457f None admin] PATH_INFO: `/v3/groups/8722ee6835904279adfb53a5649018b7` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:32.490442 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433c341-0144-4a80-aaa1-73090afd457f None admin] RBAC: Authorizing `identity:get_group(group_id=8722ee6835904279adfb53a5649018b7)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:32.491964 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433c341-0144-4a80-aaa1-73090afd457f None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:32.493781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 216/445] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 23 06:59:32 2022] GET /identity/v3/groups/8722ee6835904279adfb53a5649018b7 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:32.494340 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433c341-0144-4a80-aaa1-73090afd457f None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:32.602381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295c77d-51d7-426e-86c7-e9355709ed93 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:32.617156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433c341-0144-4a80-aaa1-73090afd457f None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:32.618817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:32.644952 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83390713-3f95-4cdf-8ead-804e191c9ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:32.648374 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83390713-3f95-4cdf-8ead-804e191c9ccc None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:32.648695 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83390713-3f95-4cdf-8ead-804e191c9ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:32.648997 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83390713-3f95-4cdf-8ead-804e191c9ccc None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:32.651238 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83390713-3f95-4cdf-8ead-804e191c9ccc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:32.651238 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83390713-3f95-4cdf-8ead-804e191c9ccc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:32.651479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83390713-3f95-4cdf-8ead-804e191c9ccc None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:32.659237 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e09cf-ec71-4bfe-876c-c0796818a728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:32.659903 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e09cf-ec71-4bfe-876c-c0796818a728 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:32.660176 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e09cf-ec71-4bfe-876c-c0796818a728 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:32.660444 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e09cf-ec71-4bfe-876c-c0796818a728 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:32.662921 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 230/446] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:32.664206 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83390713-3f95-4cdf-8ead-804e191c9ccc None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:32.712259 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:32.731689 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e09cf-ec71-4bfe-876c-c0796818a728 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:32.733162 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e09cf-ec71-4bfe-876c-c0796818a728 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:32.765993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83390713-3f95-4cdf-8ead-804e191c9ccc None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:32.800682 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b4561-24e1-4e69-89c4-eea080879227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:32.801294 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b4561-24e1-4e69-89c4-eea080879227 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:32.801573 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b4561-24e1-4e69-89c4-eea080879227 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:32.801851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b4561-24e1-4e69-89c4-eea080879227 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:32.802577 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b4561-24e1-4e69-89c4-eea080879227 None admin] RBAC: Adding query filter params (group.id=8722ee6835904279adfb53a5649018b7, role.id=7670c7c450b9404ba57e12743cb6c458, scope.project.id=e594e988024e47b4a48b43d8f828c605) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:32.802882 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b4561-24e1-4e69-89c4-eea080879227 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:32.804432 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b4561-24e1-4e69-89c4-eea080879227 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:32.810302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 231/447] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 23 06:59:32 2022] GET /identity/v3/role_assignments?group.id=8722ee6835904279adfb53a5649018b7&scope.project.id=e594e988024e47b4a48b43d8f828c605&role.id=7670c7c450b9404ba57e12743cb6c458 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:32.810916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b4561-24e1-4e69-89c4-eea080879227 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:32.841415 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 217/448] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:32.847455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e09cf-ec71-4bfe-876c-c0796818a728 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:32.849795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:32.933261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b4561-24e1-4e69-89c4-eea080879227 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:32.943990 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:32.956978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e09cf-ec71-4bfe-876c-c0796818a728 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:32.977217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297903b-6707-42d3-921f-2fcde52421d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:32.977826 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297903b-6707-42d3-921f-2fcde52421d4 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:32.978101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297903b-6707-42d3-921f-2fcde52421d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:32.978368 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297903b-6707-42d3-921f-2fcde52421d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:33.000603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d228ae-5887-4dc6-9c90-fa2cd56bc2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:33.001537 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d228ae-5887-4dc6-9c90-fa2cd56bc2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:33.001813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d228ae-5887-4dc6-9c90-fa2cd56bc2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:33.002081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d228ae-5887-4dc6-9c90-fa2cd56bc2c2 None admin] PATH_INFO: `/v3/projects/e594e988024e47b4a48b43d8f828c605` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:33.003624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d228ae-5887-4dc6-9c90-fa2cd56bc2c2 None admin] RBAC: Authorizing `identity:get_project(project_id=e594e988024e47b4a48b43d8f828c605)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:33.005154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d228ae-5887-4dc6-9c90-fa2cd56bc2c2 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:33.011822 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 218/449] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:32 2022] GET /identity/v3/projects/e594e988024e47b4a48b43d8f828c605 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:33.012299 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d228ae-5887-4dc6-9c90-fa2cd56bc2c2 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:33.071664 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297903b-6707-42d3-921f-2fcde52421d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:33.073767 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297903b-6707-42d3-921f-2fcde52421d4 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:33.093217 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:33.136032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d228ae-5887-4dc6-9c90-fa2cd56bc2c2 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:33.173315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5f0a5-9399-485e-b274-c2959c10c4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:33.173946 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5f0a5-9399-485e-b274-c2959c10c4bf None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:33.174230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5f0a5-9399-485e-b274-c2959c10c4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:33.174486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5f0a5-9399-485e-b274-c2959c10c4bf None admin] PATH_INFO: `/v3/groups/8722ee6835904279adfb53a5649018b7` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:33.175632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5f0a5-9399-485e-b274-c2959c10c4bf None admin] RBAC: Authorizing `identity:get_group(group_id=8722ee6835904279adfb53a5649018b7)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:33.177096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5f0a5-9399-485e-b274-c2959c10c4bf None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:33.178662 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 219/450] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 23 06:59:33 2022] GET /identity/v3/groups/8722ee6835904279adfb53a5649018b7 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:33.184549 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5f0a5-9399-485e-b274-c2959c10c4bf None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:33.191139 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 232/451] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:33.197291 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297903b-6707-42d3-921f-2fcde52421d4 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:33.207928 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:33.283356 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5f0a5-9399-485e-b274-c2959c10c4bf None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:33.296309 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297903b-6707-42d3-921f-2fcde52421d4 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:33.316791 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765f922-0c13-4b41-9175-315578c61dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:33.317393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765f922-0c13-4b41-9175-315578c61dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:33.317667 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765f922-0c13-4b41-9175-315578c61dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:33.317935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765f922-0c13-4b41-9175-315578c61dc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:33.318661 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b765f922-0c13-4b41-9175-315578c61dc2 None admin] RBAC: Adding query filter params (group.id=8722ee6835904279adfb53a5649018b7, role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7, scope.project.id=e594e988024e47b4a48b43d8f828c605) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:33.318968 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b765f922-0c13-4b41-9175-315578c61dc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:33.320441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b765f922-0c13-4b41-9175-315578c61dc2 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:33.326388 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 220/452] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 23 06:59:33 2022] GET /identity/v3/role_assignments?group.id=8722ee6835904279adfb53a5649018b7&scope.project.id=e594e988024e47b4a48b43d8f828c605&role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:33.330903 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f774159-8bfe-4452-ab57-d3b4ccdd0b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:33.331583 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f774159-8bfe-4452-ab57-d3b4ccdd0b05 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:33.331862 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f774159-8bfe-4452-ab57-d3b4ccdd0b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:33.332134 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f774159-8bfe-4452-ab57-d3b4ccdd0b05 None admin] PATH_INFO: `/v3/projects/c7014afbcdaa44f68b1b75e344808f41` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:33.333297 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f774159-8bfe-4452-ab57-d3b4ccdd0b05 None admin] RBAC: Authorizing `identity:get_project(project_id=c7014afbcdaa44f68b1b75e344808f41)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:33.334769 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f774159-8bfe-4452-ab57-d3b4ccdd0b05 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:33.336539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 233/453] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 06:59:33 2022] GET /identity/v3/projects/c7014afbcdaa44f68b1b75e344808f41 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:33.342382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765f922-0c13-4b41-9175-315578c61dc2 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:33.345887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:33.452629 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765f922-0c13-4b41-9175-315578c61dc2 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:33.494260 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3721d-7ff0-4183-8517-b352cc7ab981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:33.494908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3721d-7ff0-4183-8517-b352cc7ab981 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:33.495261 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3721d-7ff0-4183-8517-b352cc7ab981 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:33.495484 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3721d-7ff0-4183-8517-b352cc7ab981 None admin] PATH_INFO: `/v3/groups/3a712ed666f84aecbb18f75796c91ad3` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:33.496902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d3721d-7ff0-4183-8517-b352cc7ab981 None admin] RBAC: Authorizing `identity:get_group(group_id=3a712ed666f84aecbb18f75796c91ad3)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:33.498105 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d3721d-7ff0-4183-8517-b352cc7ab981 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:33.501214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 221/454] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Aug 23 06:59:33 2022] GET /identity/v3/groups/3a712ed666f84aecbb18f75796c91ad3 => generated 229 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:33.510695 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f774159-8bfe-4452-ab57-d3b4ccdd0b05 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:33.607458 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:33.633810 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f774159-8bfe-4452-ab57-d3b4ccdd0b05 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:33.674377 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13855e01-4c92-449d-b296-c82a1d168dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:33.675014 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13855e01-4c92-449d-b296-c82a1d168dad None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:33.675293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13855e01-4c92-449d-b296-c82a1d168dad None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:33.675604 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13855e01-4c92-449d-b296-c82a1d168dad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:33.676344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13855e01-4c92-449d-b296-c82a1d168dad None admin] RBAC: Adding query filter params (group.id=3a712ed666f84aecbb18f75796c91ad3, role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.project.id=c7014afbcdaa44f68b1b75e344808f41) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:33.676660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13855e01-4c92-449d-b296-c82a1d168dad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:33.678126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13855e01-4c92-449d-b296-c82a1d168dad None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:33.684450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 234/455] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Aug 23 06:59:33 2022] GET /identity/v3/role_assignments?group.id=3a712ed666f84aecbb18f75796c91ad3&scope.project.id=c7014afbcdaa44f68b1b75e344808f41&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:35.973966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4c285-27cd-42cc-9820-743af316933c None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:35.974460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4c285-27cd-42cc-9820-743af316933c None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:35.974460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4c285-27cd-42cc-9820-743af316933c None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:35.976077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 222/456] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:35.981135 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec4226-8bae-4ee2-8487-d9a693a182ad None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:35.981932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec4226-8bae-4ee2-8487-d9a693a182ad None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:35.982401 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec4226-8bae-4ee2-8487-d9a693a182ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:36.008443 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-45ec4226-8bae-4ee2-8487-d9a693a182ad None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:36.010039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-45ec4226-8bae-4ee2-8487-d9a693a182ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:36.105072 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 235/457] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:36.109016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96706182-3869-4114-affe-b2cb176fbca4 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:36.109301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96706182-3869-4114-affe-b2cb176fbca4 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:36.109579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96706182-3869-4114-affe-b2cb176fbca4 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:36.111966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 223/458] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:36.116777 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079fcac-4149-42f2-8b4e-169fdf9445e1 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:36.117065 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079fcac-4149-42f2-8b4e-169fdf9445e1 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:36.117339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079fcac-4149-42f2-8b4e-169fdf9445e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:36.142081 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-3079fcac-4149-42f2-8b4e-169fdf9445e1 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:36.143299 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3079fcac-4149-42f2-8b4e-169fdf9445e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:36.230647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:36.240210 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 236/459] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:36.394039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96706182-3869-4114-affe-b2cb176fbca4 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:36.484301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:36.502287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246ec29-9174-4b67-b2e1-ccedb25fb5f7 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:36.502636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246ec29-9174-4b67-b2e1-ccedb25fb5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:36.502835 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246ec29-9174-4b67-b2e1-ccedb25fb5f7 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:36.504499 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 237/460] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:36.509813 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1f71b-ca80-45db-8311-8fb2562ad578 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:36.510713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1f71b-ca80-45db-8311-8fb2562ad578 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:36.511183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1f71b-ca80-45db-8311-8fb2562ad578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:36.522738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96706182-3869-4114-affe-b2cb176fbca4 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:36.545644 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-2fe1f71b-ca80-45db-8311-8fb2562ad578 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:36.546829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-2fe1f71b-ca80-45db-8311-8fb2562ad578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:36.584515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4c2f9-cfee-499b-9260-0fcd842a4984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:36.585527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4c2f9-cfee-499b-9260-0fcd842a4984 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:36.586014 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4c2f9-cfee-499b-9260-0fcd842a4984 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:36.586471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4c2f9-cfee-499b-9260-0fcd842a4984 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:36.603228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4c2f9-cfee-499b-9260-0fcd842a4984 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:36.604612 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4c2f9-cfee-499b-9260-0fcd842a4984 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:36.621187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-3bd4c2f9-cfee-499b-9260-0fcd842a4984 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:36.621187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:36.621187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:36.621187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:36.621187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:36.621187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:36.621187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:36.621187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:36.621187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:36.621187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:36.621187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:36.621187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:36.621187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:36.621187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:36.621187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:36.621187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:36.622165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:36.622165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:36.622165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:36.622165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:36.622165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:36.622165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:36.622165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:36.622165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:36.622165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:36.622165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:36.622165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:36.622165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:36.622165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:36.622165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:36.622165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:36.622165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:36.622165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:36.623006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:36.623006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:36.623006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:36.623006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 224/461] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:36.623692 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4c2f9-cfee-499b-9260-0fcd842a4984 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:36.654717 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 238/462] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:36.656629 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1f71b-ca80-45db-8311-8fb2562ad578 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:36.778902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4c2f9-cfee-499b-9260-0fcd842a4984 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:36.823134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:36.839657 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1f71b-ca80-45db-8311-8fb2562ad578 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:36.850207 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0a48c-8133-45b1-b050-0c703cfa0d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:36.850810 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0a48c-8133-45b1-b050-0c703cfa0d1a None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:36.851087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0a48c-8133-45b1-b050-0c703cfa0d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:36.851359 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0a48c-8133-45b1-b050-0c703cfa0d1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:36.868841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0a48c-8133-45b1-b050-0c703cfa0d1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:36.870324 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0a48c-8133-45b1-b050-0c703cfa0d1a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:36.894623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-5bb0a48c-8133-45b1-b050-0c703cfa0d1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:36.894623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:36.894623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:36.894623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:36.894623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:36.894623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:36.894623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:36.894623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:36.894623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:36.894623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:36.894623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:36.894623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:36.894623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:36.894623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:36.894623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:36.894623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:36.895605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:36.895605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:36.895605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:36.895605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:36.895605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:36.895605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:36.895605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:36.895605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:36.895605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:36.895605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:36.895605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:36.895605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:36.895605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:36.895605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:36.895605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:36.895605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:36.895605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:36.896460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:36.896460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:36.896460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:36.896460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 225/463] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:36.901891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0a48c-8133-45b1-b050-0c703cfa0d1a None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:36.903030 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:36.916697 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d774f5-9aa6-4e05-a255-8abe23bfe85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:36.917320 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d774f5-9aa6-4e05-a255-8abe23bfe85a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:36.917696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d774f5-9aa6-4e05-a255-8abe23bfe85a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:36.917977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d774f5-9aa6-4e05-a255-8abe23bfe85a None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:36.918635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d774f5-9aa6-4e05-a255-8abe23bfe85a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:36.918935 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d774f5-9aa6-4e05-a255-8abe23bfe85a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:36.920453 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d774f5-9aa6-4e05-a255-8abe23bfe85a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:36.944590 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 239/464] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:36.945531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d774f5-9aa6-4e05-a255-8abe23bfe85a None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:37.017066 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0a48c-8133-45b1-b050-0c703cfa0d1a None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:37.062402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95b0ef-2099-489e-9d2f-61d92f844902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:37.063880 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95b0ef-2099-489e-9d2f-61d92f844902 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:37.064675 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95b0ef-2099-489e-9d2f-61d92f844902 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:37.065103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95b0ef-2099-489e-9d2f-61d92f844902 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:37.065919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95b0ef-2099-489e-9d2f-61d92f844902 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:37.066885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95b0ef-2099-489e-9d2f-61d92f844902 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:37.068709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95b0ef-2099-489e-9d2f-61d92f844902 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:37.078976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d774f5-9aa6-4e05-a255-8abe23bfe85a None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:37.099388 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 226/465] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:37.099388 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95b0ef-2099-489e-9d2f-61d92f844902 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:37.127853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4cc5d-07e4-4b86-8879-09c81aa9cc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:37.128869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4cc5d-07e4-4b86-8879-09c81aa9cc7f None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:37.129338 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4cc5d-07e4-4b86-8879-09c81aa9cc7f None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:37.129780 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4cc5d-07e4-4b86-8879-09c81aa9cc7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:37.135919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:37.142986 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b4cc5d-07e4-4b86-8879-09c81aa9cc7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:37.145080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b4cc5d-07e4-4b86-8879-09c81aa9cc7f None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:37.164617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-01b4cc5d-07e4-4b86-8879-09c81aa9cc7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:37.164617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:37.164617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:37.164617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:37.164617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:37.164617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:37.164617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:37.164617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:37.164617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:37.164617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:37.164617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:37.164617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:37.164617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:37.164617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:37.164617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:37.164617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:37.165727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:37.165727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:37.165727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:37.165727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:37.165727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:37.165727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:37.165727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:37.165727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:37.165727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:37.165727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:37.165727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:37.165727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:37.165727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:37.165727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:37.165727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:37.165727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:37.165727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:37.166798 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:37.166798 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:37.166798 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:37.169116 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 240/466] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:37.170438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47d659-80ac-43bc-b5e7-a5690bc73c0e None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:37.170853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47d659-80ac-43bc-b5e7-a5690bc73c0e None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:37.171431 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47d659-80ac-43bc-b5e7-a5690bc73c0e None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:37.177358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 241/467] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:37.177358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47d659-80ac-43bc-b5e7-a5690bc73c0e None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:37.213853 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95b0ef-2099-489e-9d2f-61d92f844902 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:37.223689 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:37.253464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455db7f-230e-4b78-8366-709fa0ee8484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:37.254312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455db7f-230e-4b78-8366-709fa0ee8484 None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:37.254727 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455db7f-230e-4b78-8366-709fa0ee8484 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:37.255136 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455db7f-230e-4b78-8366-709fa0ee8484 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:37.256023 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455db7f-230e-4b78-8366-709fa0ee8484 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:37.257721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455db7f-230e-4b78-8366-709fa0ee8484 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:37.282650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 227/468] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 23 06:59:37 2022] POST /identity/v3/users => generated 246 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:59:37.283747 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455db7f-230e-4b78-8366-709fa0ee8484 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:37.294291 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47d659-80ac-43bc-b5e7-a5690bc73c0e None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:37.348443 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3762b-ba6d-4316-a06c-598fc4e3b547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:37.349589 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3762b-ba6d-4316-a06c-598fc4e3b547 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:37.350014 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3762b-ba6d-4316-a06c-598fc4e3b547 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:37.350426 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3762b-ba6d-4316-a06c-598fc4e3b547 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:37.351991 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3762b-ba6d-4316-a06c-598fc4e3b547 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:37.354304 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3762b-ba6d-4316-a06c-598fc4e3b547 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:37.390965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 242/469] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 23 06:59:37 2022] POST /identity/v3/users => generated 248 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:59:37.393538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0111a49-3aa7-4390-a679-a3cf6ff9fbee None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:37.394717 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0111a49-3aa7-4390-a679-a3cf6ff9fbee None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:37.395162 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0111a49-3aa7-4390-a679-a3cf6ff9fbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:37.397755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:37.413289 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455db7f-230e-4b78-8366-709fa0ee8484 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:37.429041 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-a0111a49-3aa7-4390-a679-a3cf6ff9fbee None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:37.431266 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a0111a49-3aa7-4390-a679-a3cf6ff9fbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:37.457182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea003861-decb-4e03-bf74-1f43f047e542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:37.457876 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea003861-decb-4e03-bf74-1f43f047e542 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:37.458156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea003861-decb-4e03-bf74-1f43f047e542 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:37.458425 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea003861-decb-4e03-bf74-1f43f047e542 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:37.459082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea003861-decb-4e03-bf74-1f43f047e542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:37.459381 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea003861-decb-4e03-bf74-1f43f047e542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:37.460919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea003861-decb-4e03-bf74-1f43f047e542 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:37.475157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 228/470] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:37.481206 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea003861-decb-4e03-bf74-1f43f047e542 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:37.537129 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 243/471] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:37.558976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87786c-f26a-40bb-80b2-386ae1ed9c36 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:37.559527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87786c-f26a-40bb-80b2-386ae1ed9c36 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:37.560494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87786c-f26a-40bb-80b2-386ae1ed9c36 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:37.562432 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 244/472] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:37.569537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc632c-4091-4ca3-b6eb-6db180ee72c3 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:37.570042 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc632c-4091-4ca3-b6eb-6db180ee72c3 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:37.570471 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc632c-4091-4ca3-b6eb-6db180ee72c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:37.591987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-d4fc632c-4091-4ca3-b6eb-6db180ee72c3 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:37.593433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d4fc632c-4091-4ca3-b6eb-6db180ee72c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:37.616468 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea003861-decb-4e03-bf74-1f43f047e542 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:37.628847 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:37.672233 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e766e-b525-4c16-8c70-b92476d3371d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:37.672420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e766e-b525-4c16-8c70-b92476d3371d None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:37.672689 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e766e-b525-4c16-8c70-b92476d3371d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:37.673065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e766e-b525-4c16-8c70-b92476d3371d None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:37.673844 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e766e-b525-4c16-8c70-b92476d3371d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:37.675547 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e766e-b525-4c16-8c70-b92476d3371d None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:37.692933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 245/473] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:59:37.705599 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 229/474] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 23 06:59:37 2022] POST /identity/v3/users => generated 248 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 06:59:37.840403 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc632c-4091-4ca3-b6eb-6db180ee72c3 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:37.951673 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc632c-4091-4ca3-b6eb-6db180ee72c3 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:37.976517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e766e-b525-4c16-8c70-b92476d3371d None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:37.986951 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:37.989977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd342b69-7531-4270-9bb0-e241b9941590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:37.990862 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd342b69-7531-4270-9bb0-e241b9941590 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:37.991285 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd342b69-7531-4270-9bb0-e241b9941590 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:37.991599 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd342b69-7531-4270-9bb0-e241b9941590 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:38.003389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd342b69-7531-4270-9bb0-e241b9941590 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:38.004936 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd342b69-7531-4270-9bb0-e241b9941590 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:38.021402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-bd342b69-7531-4270-9bb0-e241b9941590 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:38.021402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:38.021402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:38.021402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:38.021402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:38.021402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:38.021402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:38.021402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:38.021402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:38.021402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:38.021402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:38.021402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:38.021402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:38.021402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:38.021402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:38.021402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:38.022355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:38.022355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:38.022355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:38.022355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:38.022355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:38.022355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:38.022355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:38.022355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:38.022355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:38.022355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:38.022355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:38.022355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:38.022355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:38.022355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:38.022355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:38.022355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:38.022355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:38.023179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:38.023179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:38.023179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:38.023179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 246/475] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:38.028302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd342b69-7531-4270-9bb0-e241b9941590 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:38.044794 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:38.101800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e766e-b525-4c16-8c70-b92476d3371d None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:38.148689 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0f7e7-d4ea-4e6a-807b-fcb86726679c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:38.149332 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0f7e7-d4ea-4e6a-807b-fcb86726679c None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:38.149834 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0f7e7-d4ea-4e6a-807b-fcb86726679c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:38.150117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0f7e7-d4ea-4e6a-807b-fcb86726679c None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:38.151254 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0f7e7-d4ea-4e6a-807b-fcb86726679c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:38.153007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0f7e7-d4ea-4e6a-807b-fcb86726679c None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:38.154975 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd342b69-7531-4270-9bb0-e241b9941590 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:38.164419 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.sql.core [None req-9fd0f7e7-d4ea-4e6a-807b-fcb86726679c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 06:59:38.164419 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 23 06:59:38.164419 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [parameters: {'id': '5c89f0b680b74b389cf8fa06fffc23c9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 23 06:59:38.164419 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81714) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 06:59:38.182736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-9fd0f7e7-d4ea-4e6a-807b-fcb86726679c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 23 06:59:38.182736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:38.182736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 06:59:38.182736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 06:59:38.182736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 06:59:38.182736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 06:59:38.182736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 06:59:38.182736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 06:59:38.182736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 06:59:38.182736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application conn.query(q) Aug 23 06:59:38.182736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 06:59:38.182736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 06:59:38.182736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 06:59:38.182736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application result.read() Aug 23 06:59:38.182736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 06:59:38.183838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 06:59:38.183838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 06:59:38.183838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 06:59:38.183838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 06:59:38.183838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 06:59:38.183838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 06:59:38.183838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 06:59:38.183838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 06:59:38.183838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:38.183838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 06:59:38.183838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:38.183838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:38.183838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 06:59:38.183838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 06:59:38.183838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 23 06:59:38.183838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return ref.to_dict() Aug 23 06:59:38.183838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 06:59:38.183838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application next(self.gen) Aug 23 06:59:38.184730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 06:59:38.184730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application yield resource Aug 23 06:59:38.184730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 06:59:38.184730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application next(self.gen) Aug 23 06:59:38.184730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 06:59:38.184730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 06:59:38.184730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 06:59:38.184730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 06:59:38.184730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 06:59:38.184730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise self.value Aug 23 06:59:38.184730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 06:59:38.184730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 06:59:38.184730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 06:59:38.184730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application session.commit() Aug 23 06:59:38.184730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 06:59:38.184730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 06:59:38.184730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 06:59:38.185575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 06:59:38.185575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 06:59:38.185575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self.session.flush() Aug 23 06:59:38.185575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 06:59:38.185575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 06:59:38.185575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 06:59:38.185575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 06:59:38.185575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 06:59:38.185575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application compat.raise_( Aug 23 06:59:38.185575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 06:59:38.185575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception Aug 23 06:59:38.185575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 06:59:38.185575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 06:59:38.185575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 06:59:38.185575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 06:59:38.185575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 06:59:38.185575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 06:59:38.186424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 06:59:38.186424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 06:59:38.186424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 23 06:59:38.186424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 23 06:59:38.186424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 06:59:38.186424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 06:59:38.186424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 06:59:38.186424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 06:59:38.186424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 06:59:38.186424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 06:59:38.186424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 06:59:38.186424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 06:59:38.186424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 06:59:38.186424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 06:59:38.186424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 06:59:38.186424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception Aug 23 06:59:38.187227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 06:59:38.187227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 06:59:38.187227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 06:59:38.187227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 06:59:38.187227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 06:59:38.187227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 06:59:38.187227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 06:59:38.187227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application conn.query(q) Aug 23 06:59:38.187227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 06:59:38.187227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 06:59:38.187227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 06:59:38.187227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application result.read() Aug 23 06:59:38.187227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 06:59:38.187227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 06:59:38.187227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 06:59:38.187227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 06:59:38.188078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 06:59:38.188078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 06:59:38.188078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 06:59:38.188078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 06:59:38.188078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 06:59:38.188078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 23 06:59:38.188078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [parameters: {'id': '5c89f0b680b74b389cf8fa06fffc23c9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 23 06:59:38.188078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 06:59:38.188078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:38.188078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:59:38.188078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:38.188078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:38.188078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:38.188078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:38.188078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:38.188078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:38.189053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:38.189053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:38.189053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:38.189053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:38.189053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:38.189053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:38.189053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 23 06:59:38.189053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 23 06:59:38.189053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:38.189053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:38.189053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 23 06:59:38.189053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 23 06:59:38.189053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 06:59:38.189053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 06:59:38.189053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 23 06:59:38.189053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:38.192844 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 230/476] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Aug 23 06:59:37 2022] POST /identity/v3/roles => generated 144 bytes in 217 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:38.199343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0f7e7-d4ea-4e6a-807b-fcb86726679c None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:38.199973 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43209548-c4e6-49a8-9854-083b88f6cc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:38.200661 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43209548-c4e6-49a8-9854-083b88f6cc03 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:38.200882 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43209548-c4e6-49a8-9854-083b88f6cc03 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:38.201150 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43209548-c4e6-49a8-9854-083b88f6cc03 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:38.201951 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43209548-c4e6-49a8-9854-083b88f6cc03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:38.202192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43209548-c4e6-49a8-9854-083b88f6cc03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:38.203772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43209548-c4e6-49a8-9854-083b88f6cc03 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:38.223462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 247/477] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:38.226746 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43209548-c4e6-49a8-9854-083b88f6cc03 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:38.331819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:38.370237 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0f7e7-d4ea-4e6a-807b-fcb86726679c None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:38.374753 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:38.407336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f290df-5729-4236-9f5b-0548be3bd16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:38.408410 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f290df-5729-4236-9f5b-0548be3bd16b None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:38.408873 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f290df-5729-4236-9f5b-0548be3bd16b None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:38.409279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f290df-5729-4236-9f5b-0548be3bd16b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:38.420195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f290df-5729-4236-9f5b-0548be3bd16b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:38.425832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-22f290df-5729-4236-9f5b-0548be3bd16b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:59:38.425832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:38.425832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:38.425832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:38.425832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:38.425832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:38.425832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:38.425832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:38.425832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:38.425832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:38.425832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:38.425832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:38.425832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:38.425832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:38.425832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:38.425832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:38.426881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:38.426881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:38.426881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:38.426881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:38.426881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:38.426881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:38.426881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:38.426881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:38.426881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:38.426881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:38.426881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:38.426881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:38.426881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:38.426881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:38.426881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:38.426881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:38.427901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:38.427901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:38.427901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:38.427901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:38.427901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:38.427901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:38.427901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:38.427901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:38.427901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:38.427901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:38.427901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:59:38.427901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:38.430628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 231/478] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 06:59:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:38.437558 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f290df-5729-4236-9f5b-0548be3bd16b None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:38.459560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43209548-c4e6-49a8-9854-083b88f6cc03 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:38.519274 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccfe69c-dd53-4e56-be17-f6ec2bc1b8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:38.519914 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfe69c-dd53-4e56-be17-f6ec2bc1b8a1 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:38.520189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfe69c-dd53-4e56-be17-f6ec2bc1b8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:38.520457 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfe69c-dd53-4e56-be17-f6ec2bc1b8a1 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:38.521105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccfe69c-dd53-4e56-be17-f6ec2bc1b8a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:38.523947 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccfe69c-dd53-4e56-be17-f6ec2bc1b8a1 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:38.574667 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 248/479] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 23 06:59:38 2022] POST /identity/v3/users => generated 249 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:59:38.614630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f290df-5729-4236-9f5b-0548be3bd16b None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:38.661339 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:38.673752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ca68c-4d80-42cd-b99a-709209f9b089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:38.674395 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ca68c-4d80-42cd-b99a-709209f9b089 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:38.674677 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ca68c-4d80-42cd-b99a-709209f9b089 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:38.675005 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ca68c-4d80-42cd-b99a-709209f9b089 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:38.675739 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ca68c-4d80-42cd-b99a-709209f9b089 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:38.679911 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ca68c-4d80-42cd-b99a-709209f9b089 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:38.681420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ca68c-4d80-42cd-b99a-709209f9b089 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:38.691440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 232/480] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:39.718902 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208349b-c926-4558-8228-492118af57de None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:39.719461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208349b-c926-4558-8228-492118af57de None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:39.719461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208349b-c926-4558-8228-492118af57de None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:39.721109 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 249/481] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:39.726324 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd097372-1cdb-470b-a8e1-55e988ecfea2 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:39.726722 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd097372-1cdb-470b-a8e1-55e988ecfea2 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:39.726897 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd097372-1cdb-470b-a8e1-55e988ecfea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:39.750348 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-fd097372-1cdb-470b-a8e1-55e988ecfea2 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:39.751670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-fd097372-1cdb-470b-a8e1-55e988ecfea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:39.834760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2aab7-401f-4dcc-b2da-96d068eb843f None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:39.835106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2aab7-401f-4dcc-b2da-96d068eb843f None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:39.835302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2aab7-401f-4dcc-b2da-96d068eb843f None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:39.836860 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 250/482] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:59:39.844106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7fa45-b0a8-44be-9dc3-424098751021 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:39.844386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7fa45-b0a8-44be-9dc3-424098751021 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:39.844662 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7fa45-b0a8-44be-9dc3-424098751021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:39.846493 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 233/483] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:39.867954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-13b7fa45-b0a8-44be-9dc3-424098751021 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:39.869245 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-13b7fa45-b0a8-44be-9dc3-424098751021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:39.936801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 251/484] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:59:40.173445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd097372-1cdb-470b-a8e1-55e988ecfea2 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:40.292628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd097372-1cdb-470b-a8e1-55e988ecfea2 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:40.321985 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:40.336977 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cb05a-426d-42dd-9e2f-fa1883e54741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:40.337721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cb05a-426d-42dd-9e2f-fa1883e54741 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:40.338154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cb05a-426d-42dd-9e2f-fa1883e54741 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:40.338691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cb05a-426d-42dd-9e2f-fa1883e54741 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:40.353231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1cb05a-426d-42dd-9e2f-fa1883e54741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:40.357945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-da1cb05a-426d-42dd-9e2f-fa1883e54741 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:40.357945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:40.357945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:40.357945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:40.357945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:40.357945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:40.357945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:40.357945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:40.357945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:40.357945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:40.357945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:40.357945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:40.357945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:40.357945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:40.357945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:40.357945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:40.358815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:40.358815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:40.358815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:40.358815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:40.358815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:40.358815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:40.358815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:40.358815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:40.358815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:40.358815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:40.358815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:40.358815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:40.358815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:40.358815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:40.358815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:40.358815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:40.359642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:40.359642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:40.359642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:40.359642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:40.359642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:40.359642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:40.359642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:40.359642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:40.359642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:40.359642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:40.359642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:40.359642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:40.359642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 234/485] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 06:59:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:40.365434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7fa45-b0a8-44be-9dc3-424098751021 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:40.444531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:40.496165 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7fa45-b0a8-44be-9dc3-424098751021 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:40.506126 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cb05a-426d-42dd-9e2f-fa1883e54741 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:40.537102 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5fc73-a99c-4e50-a210-f3c88755751c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:40.537855 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5fc73-a99c-4e50-a210-f3c88755751c None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:40.538289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5fc73-a99c-4e50-a210-f3c88755751c None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:40.538544 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5fc73-a99c-4e50-a210-f3c88755751c None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:40.539382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5fc73-a99c-4e50-a210-f3c88755751c None admin] RBAC: Adding query filter params (name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:40.539866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5fc73-a99c-4e50-a210-f3c88755751c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:40.541315 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5fc73-a99c-4e50-a210-f3c88755751c None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:40.551195 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 252/486] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 06:59:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:40.552491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa58ad2-ffa9-49a0-8191-97841dafbdb4 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:40.552776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa58ad2-ffa9-49a0-8191-97841dafbdb4 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:40.553050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa58ad2-ffa9-49a0-8191-97841dafbdb4 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:40.554559 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 253/487] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:40.556500 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa58ad2-ffa9-49a0-8191-97841dafbdb4 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:40.616720 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cb05a-426d-42dd-9e2f-fa1883e54741 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:40.656952 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31475ea-c959-4587-b2cd-9ad0cfbec9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:40.657603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31475ea-c959-4587-b2cd-9ad0cfbec9fe None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:40.657918 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31475ea-c959-4587-b2cd-9ad0cfbec9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:40.658187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31475ea-c959-4587-b2cd-9ad0cfbec9fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:40.671555 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31475ea-c959-4587-b2cd-9ad0cfbec9fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:40.677956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-d31475ea-c959-4587-b2cd-9ad0cfbec9fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:40.677956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:40.677956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:40.677956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:40.677956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:40.677956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:40.677956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:40.677956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:40.677956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:40.677956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:40.677956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:40.677956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:40.677956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:40.677956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:40.677956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:40.677956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:40.678872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:40.678872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:40.678872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:40.678872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:40.678872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:40.678872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:40.678872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:40.678872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:40.678872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:40.678872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:40.678872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:40.678872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:40.678872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:40.678872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:40.678872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:40.678872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:40.679690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:40.679690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:40.679690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:40.679690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:40.679690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:40.679690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:40.679690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:40.679690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:40.679690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:40.679690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:40.679690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:40.679690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:40.679690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 235/488] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 06:59:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:40.680784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a9421-c6cf-479f-bd8e-30c8ce706500 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:40.680913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a9421-c6cf-479f-bd8e-30c8ce706500 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:40.681186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a9421-c6cf-479f-bd8e-30c8ce706500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:40.694794 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa58ad2-ffa9-49a0-8191-97841dafbdb4 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:40.705992 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-0e8a9421-c6cf-479f-bd8e-30c8ce706500 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:40.707654 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-0e8a9421-c6cf-479f-bd8e-30c8ce706500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:40.717061 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:40.739600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cef38-f87f-4f91-adbc-bd03e3b43735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:40.741564 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cef38-f87f-4f91-adbc-bd03e3b43735 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:40.741991 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cef38-f87f-4f91-adbc-bd03e3b43735 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:40.742653 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cef38-f87f-4f91-adbc-bd03e3b43735 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:40.756527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:40.759238 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3cef38-f87f-4f91-adbc-bd03e3b43735 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:40.760967 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3cef38-f87f-4f91-adbc-bd03e3b43735 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:40.780331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-1a3cef38-f87f-4f91-adbc-bd03e3b43735 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:40.780331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:40.780331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:40.780331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:40.780331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:40.780331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:40.780331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:40.780331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:40.780331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:40.780331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:40.780331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:40.780331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:40.780331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:40.780331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:40.780331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:40.780331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:40.781385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:40.781385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:40.781385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:40.781385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:40.781385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:40.781385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:40.781385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:40.781385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:40.781385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:40.781385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:40.781385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:40.781385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:40.781385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:40.781385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:40.781385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:40.781385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:40.781385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:40.782215 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:40.782215 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:40.782215 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:40.782215 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 254/489] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:40.783460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cef38-f87f-4f91-adbc-bd03e3b43735 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:40.800173 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 236/490] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:40.801328 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a9421-c6cf-479f-bd8e-30c8ce706500 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:40.919651 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a9421-c6cf-479f-bd8e-30c8ce706500 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:40.957903 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648644a-ce9a-4e8a-843e-267a64819605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:40.959601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648644a-ce9a-4e8a-843e-267a64819605 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:40.960078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648644a-ce9a-4e8a-843e-267a64819605 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:40.961082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648644a-ce9a-4e8a-843e-267a64819605 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:40.962057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648644a-ce9a-4e8a-843e-267a64819605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:40.964063 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cef38-f87f-4f91-adbc-bd03e3b43735 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:40.965962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648644a-ce9a-4e8a-843e-267a64819605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:40.967569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648644a-ce9a-4e8a-843e-267a64819605 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:40.990312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 237/491] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:40.991879 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06346ab-9cee-4b3e-abae-a34ec99b03c7 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:40.992396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06346ab-9cee-4b3e-abae-a34ec99b03c7 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:40.992865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06346ab-9cee-4b3e-abae-a34ec99b03c7 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:40.994243 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 238/492] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:40.999759 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3c5e3-798d-4dac-af89-95dc26b95965 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:41.000236 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3c5e3-798d-4dac-af89-95dc26b95965 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:41.000675 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3c5e3-798d-4dac-af89-95dc26b95965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:41.019522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ff809-1b84-4e1a-9dd4-630186b57408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:41.020175 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ff809-1b84-4e1a-9dd4-630186b57408 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:41.020509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ff809-1b84-4e1a-9dd4-630186b57408 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:41.020782 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ff809-1b84-4e1a-9dd4-630186b57408 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:41.021449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ff809-1b84-4e1a-9dd4-630186b57408 None admin] RBAC: Adding query filter params (name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:41.021799 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ff809-1b84-4e1a-9dd4-630186b57408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:41.022108 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-47f3c5e3-798d-4dac-af89-95dc26b95965 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:41.023299 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ff809-1b84-4e1a-9dd4-630186b57408 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:41.023663 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-47f3c5e3-798d-4dac-af89-95dc26b95965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:41.039320 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 255/493] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 06:59:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:41.040590 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ff809-1b84-4e1a-9dd4-630186b57408 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:41.056185 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:41.086077 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:41.105007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 239/494] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:41.106322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3c5e3-798d-4dac-af89-95dc26b95965 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:41.169845 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ff809-1b84-4e1a-9dd4-630186b57408 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:41.208886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e684620-8deb-457d-a855-943f97b82fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:41.209514 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e684620-8deb-457d-a855-943f97b82fdf None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:41.209796 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e684620-8deb-457d-a855-943f97b82fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:41.210068 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e684620-8deb-457d-a855-943f97b82fdf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:41.217901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e684620-8deb-457d-a855-943f97b82fdf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:41.219403 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e684620-8deb-457d-a855-943f97b82fdf None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:41.222774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3c5e3-798d-4dac-af89-95dc26b95965 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:41.232396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-3e684620-8deb-457d-a855-943f97b82fdf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:59:41.232396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:41.232396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:41.232396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:41.232396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:41.232396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:41.232396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:41.232396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:41.232396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:41.232396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:41.232396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:41.232396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:41.232396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:59:41.232396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:59:41.232396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:59:41.232396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:59:41.233501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:41.233501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:41.233501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:59:41.233501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:41.233501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:59:41.233501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:41.233501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:41.233501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:41.233501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:41.233501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:41.233501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:41.233501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:41.233501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:41.233501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:41.233501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:41.233501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:41.233501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:41.233501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:41.235196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:41.235196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:41.235196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:59:41.235196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:59:41.235196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:59:41.235196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:59:41.235196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:59:41.235196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:59:41.235196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:59:41.235196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:41.235196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 256/495] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Aug 23 06:59:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:41.235196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0b46b-8516-4938-94bb-84b1af253255 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:41.235196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0b46b-8516-4938-94bb-84b1af253255 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:41.236288 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0b46b-8516-4938-94bb-84b1af253255 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:41.240800 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 257/496] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:41.241609 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0b46b-8516-4938-94bb-84b1af253255 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:41.269425 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47bbd5-3203-4772-93df-cdd036d6ed68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:41.270106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47bbd5-3203-4772-93df-cdd036d6ed68 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:41.270363 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47bbd5-3203-4772-93df-cdd036d6ed68 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:41.270594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47bbd5-3203-4772-93df-cdd036d6ed68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:41.280681 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47bbd5-3203-4772-93df-cdd036d6ed68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:41.282155 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47bbd5-3203-4772-93df-cdd036d6ed68 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:41.296962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-7a47bbd5-3203-4772-93df-cdd036d6ed68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:41.296962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:41.296962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:41.296962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:41.296962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:41.296962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:41.296962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:41.296962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:41.296962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:41.296962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:41.296962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:41.296962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:41.296962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:41.296962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:41.296962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:41.296962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:41.297865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:41.297865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:41.297865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:41.297865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:41.297865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:41.297865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:41.297865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:41.297865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:41.297865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:41.297865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:41.297865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:41.297865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:41.297865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:41.297865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:41.297865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:41.297865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:41.297865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:41.298751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:41.298751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:41.298751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:41.298751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 240/497] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:41.298987 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47bbd5-3203-4772-93df-cdd036d6ed68 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:41.316344 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:41.352819 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:41.364551 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0b46b-8516-4938-94bb-84b1af253255 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:41.399854 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36111ce1-1469-4ec3-addd-01b77f1ee291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:41.400482 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36111ce1-1469-4ec3-addd-01b77f1ee291 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:41.400755 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36111ce1-1469-4ec3-addd-01b77f1ee291 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:41.401028 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36111ce1-1469-4ec3-addd-01b77f1ee291 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:41.406539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47bbd5-3203-4772-93df-cdd036d6ed68 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:41.411462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36111ce1-1469-4ec3-addd-01b77f1ee291 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:41.416149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-36111ce1-1469-4ec3-addd-01b77f1ee291 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:41.416149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:41.416149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:41.416149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:41.416149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:41.416149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:41.416149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:41.416149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:41.416149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:41.416149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:41.416149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:41.416149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:41.416149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:41.416149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:41.416149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:41.416149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:41.417110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:41.417110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:41.417110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:41.417110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:41.417110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:41.417110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:41.417110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:41.417110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:41.417110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:41.417110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:41.417110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:41.417110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:41.417110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:41.417110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:41.417110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:41.417110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:41.417895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:41.417895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:41.417895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:41.417895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:41.417895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:41.417895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:41.417895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:41.417895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:41.417895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:41.417895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:41.417895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:41.417895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:41.417895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 258/498] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 06:59:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:41.418546 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae137c7-47c4-4095-b7dd-c35ac25f737c None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:41.418610 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae137c7-47c4-4095-b7dd-c35ac25f737c None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:41.418872 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae137c7-47c4-4095-b7dd-c35ac25f737c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:41.436459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-8ae137c7-47c4-4095-b7dd-c35ac25f737c None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:41.437667 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8ae137c7-47c4-4095-b7dd-c35ac25f737c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:41.446933 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260177c6-6f92-4577-b944-e567ccc99f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:41.447590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260177c6-6f92-4577-b944-e567ccc99f3d None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:41.448007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260177c6-6f92-4577-b944-e567ccc99f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:41.448282 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260177c6-6f92-4577-b944-e567ccc99f3d None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:41.449018 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260177c6-6f92-4577-b944-e567ccc99f3d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:41.449316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260177c6-6f92-4577-b944-e567ccc99f3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:41.451080 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260177c6-6f92-4577-b944-e567ccc99f3d None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:41.491162 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 241/499] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Aug 23 06:59:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:41.491162 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260177c6-6f92-4577-b944-e567ccc99f3d None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:41.503574 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 259/500] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:41.504413 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae137c7-47c4-4095-b7dd-c35ac25f737c None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:41.565760 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:41.602539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260177c6-6f92-4577-b944-e567ccc99f3d None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:41.606789 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:41.627757 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae137c7-47c4-4095-b7dd-c35ac25f737c None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:41.644062 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd41d8-7c46-41f3-867e-dd91715b5bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:41.644670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd41d8-7c46-41f3-867e-dd91715b5bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:41.644950 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd41d8-7c46-41f3-867e-dd91715b5bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:41.645223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd41d8-7c46-41f3-867e-dd91715b5bf6 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:41.645883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd41d8-7c46-41f3-867e-dd91715b5bf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:41.646174 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd41d8-7c46-41f3-867e-dd91715b5bf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:41.647602 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd41d8-7c46-41f3-867e-dd91715b5bf6 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:41.663918 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 242/501] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:41.663918 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd41d8-7c46-41f3-867e-dd91715b5bf6 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:41.672169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf90b9-5766-4d54-9b2d-b1eb775c31c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:41.672798 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf90b9-5766-4d54-9b2d-b1eb775c31c4 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:41.673071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf90b9-5766-4d54-9b2d-b1eb775c31c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:41.673343 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf90b9-5766-4d54-9b2d-b1eb775c31c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:41.686327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf90b9-5766-4d54-9b2d-b1eb775c31c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:41.687836 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf90b9-5766-4d54-9b2d-b1eb775c31c4 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:41.705995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-79cf90b9-5766-4d54-9b2d-b1eb775c31c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:41.705995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:41.705995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:41.705995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:41.705995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:41.705995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:41.705995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:41.705995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:41.705995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:41.705995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:41.705995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:41.705995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:41.705995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:41.705995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:41.705995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:41.705995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:41.706913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:41.706913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:41.706913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:41.706913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:41.706913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:41.706913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:41.706913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:41.706913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:41.706913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:41.706913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:41.706913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:41.706913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:41.706913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:41.706913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:41.706913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:41.706913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:41.706913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:41.707866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:41.707866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:41.707866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:41.711451 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 260/502] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:41.711451 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf90b9-5766-4d54-9b2d-b1eb775c31c4 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:41.773512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd41d8-7c46-41f3-867e-dd91715b5bf6 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:41.803562 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:41.809122 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623fd9e-8319-4eda-b877-5495ee7349ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:41.809758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623fd9e-8319-4eda-b877-5495ee7349ff None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:41.810036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623fd9e-8319-4eda-b877-5495ee7349ff None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:41.810307 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623fd9e-8319-4eda-b877-5495ee7349ff None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:41.810972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623fd9e-8319-4eda-b877-5495ee7349ff None admin] RBAC: Adding query filter params (name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:41.811273 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623fd9e-8319-4eda-b877-5495ee7349ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:41.812706 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623fd9e-8319-4eda-b877-5495ee7349ff None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:41.814582 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf90b9-5766-4d54-9b2d-b1eb775c31c4 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:41.821004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 243/503] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 06:59:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:41.821785 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623fd9e-8319-4eda-b877-5495ee7349ff None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:41.853458 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7cbfb-f1af-4111-b5de-bb9ad6efe54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:41.853977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7cbfb-f1af-4111-b5de-bb9ad6efe54b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:41.854251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7cbfb-f1af-4111-b5de-bb9ad6efe54b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:41.854529 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7cbfb-f1af-4111-b5de-bb9ad6efe54b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:41.857153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:41.925766 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7cbfb-f1af-4111-b5de-bb9ad6efe54b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:41.927121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623fd9e-8319-4eda-b877-5495ee7349ff None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:41.927228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7cbfb-f1af-4111-b5de-bb9ad6efe54b None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:41.965693 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebaeea2-53ae-44e0-a481-db1c8cc2b322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:41.966306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebaeea2-53ae-44e0-a481-db1c8cc2b322 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:41.966583 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebaeea2-53ae-44e0-a481-db1c8cc2b322 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:41.966851 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebaeea2-53ae-44e0-a481-db1c8cc2b322 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:41.974536 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebaeea2-53ae-44e0-a481-db1c8cc2b322 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:41.976069 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebaeea2-53ae-44e0-a481-db1c8cc2b322 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:41.988310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-aebaeea2-53ae-44e0-a481-db1c8cc2b322 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 06:59:41.988310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:41.988310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:41.988310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:41.988310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:41.988310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:41.988310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:41.988310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:41.988310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:41.988310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:41.988310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:41.988310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:41.988310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:59:41.988310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:59:41.988310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:59:41.988310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:59:41.989413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:41.989413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:41.989413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:59:41.989413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:41.989413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:59:41.989413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:41.989413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:41.989413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:41.989413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:41.989413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:41.989413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:41.989413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:41.989413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:41.989413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:41.989413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:41.989413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:41.989413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:41.989413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:41.990322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:41.990322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:41.990322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:59:41.990322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:59:41.990322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:59:41.990322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:59:41.990322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:59:41.990322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:59:41.990322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 06:59:41.990322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:41.990322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 244/504] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:41.990322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebaeea2-53ae-44e0-a481-db1c8cc2b322 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:42.027482 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 261/505] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:42.028329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7cbfb-f1af-4111-b5de-bb9ad6efe54b None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:42.052693 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:42.061291 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:42.101270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebaeea2-53ae-44e0-a481-db1c8cc2b322 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:42.127374 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7cbfb-f1af-4111-b5de-bb9ad6efe54b None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:42.135885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d890ce-e841-4a71-9343-e8b59f50445c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:42.136498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d890ce-e841-4a71-9343-e8b59f50445c None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:42.137107 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d890ce-e841-4a71-9343-e8b59f50445c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:42.137372 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d890ce-e841-4a71-9343-e8b59f50445c None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:42.138334 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d890ce-e841-4a71-9343-e8b59f50445c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:42.138641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d890ce-e841-4a71-9343-e8b59f50445c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:42.140070 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d890ce-e841-4a71-9343-e8b59f50445c None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:42.152969 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 245/506] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:42.153786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d890ce-e841-4a71-9343-e8b59f50445c None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:42.161231 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2eadd-56b0-49d0-9ba2-a191b55b2a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:42.161865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2eadd-56b0-49d0-9ba2-a191b55b2a5a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:42.162142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2eadd-56b0-49d0-9ba2-a191b55b2a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:42.162428 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2eadd-56b0-49d0-9ba2-a191b55b2a5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:42.173459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2eadd-56b0-49d0-9ba2-a191b55b2a5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:42.178145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-05f2eadd-56b0-49d0-9ba2-a191b55b2a5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:42.178145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:42.178145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:42.178145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:42.178145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:42.178145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:42.178145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:42.178145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:42.178145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:42.178145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:42.178145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:42.178145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:42.178145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:42.178145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:42.178145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:42.178145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:42.179093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:42.179093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:42.179093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:42.179093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:42.179093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:42.179093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:42.179093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:42.179093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:42.179093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:42.179093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:42.179093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:42.179093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:42.179093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:42.179093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:42.179093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:42.179093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:42.179929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:42.179929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:42.179929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:42.179929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:42.179929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:42.179929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:42.179929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:42.179929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:42.179929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:42.179929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:42.179929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:42.179929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:42.179929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 262/507] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 06:59:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:42.180597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2eadd-56b0-49d0-9ba2-a191b55b2a5a None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:42.256941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d890ce-e841-4a71-9343-e8b59f50445c None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:42.270979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:42.272389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:42.280151 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2eadd-56b0-49d0-9ba2-a191b55b2a5a None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:42.292283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384de145-6a84-416d-b293-ec0a99d350d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:42.292906 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384de145-6a84-416d-b293-ec0a99d350d2 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:42.293240 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384de145-6a84-416d-b293-ec0a99d350d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:42.293509 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384de145-6a84-416d-b293-ec0a99d350d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:42.303735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384de145-6a84-416d-b293-ec0a99d350d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:42.305275 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384de145-6a84-416d-b293-ec0a99d350d2 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:42.315235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb629f-8688-48c9-9f1f-25665c6e58ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:42.315860 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb629f-8688-48c9-9f1f-25665c6e58ef None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:42.316139 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb629f-8688-48c9-9f1f-25665c6e58ef None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:42.316405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb629f-8688-48c9-9f1f-25665c6e58ef None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:42.317124 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb629f-8688-48c9-9f1f-25665c6e58ef None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:42.317419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb629f-8688-48c9-9f1f-25665c6e58ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:42.318856 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb629f-8688-48c9-9f1f-25665c6e58ef None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:42.319450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-384de145-6a84-416d-b293-ec0a99d350d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:42.319450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:42.319450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:42.319450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:42.319450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:42.319450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:42.319450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:42.319450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:42.319450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:42.319450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:42.319450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:42.319450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:42.319450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:42.319450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:42.319450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:42.319450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:42.320462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:42.320462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:42.320462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:42.320462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:42.320462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:42.320462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:42.320462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:42.320462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:42.320462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:42.320462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:42.320462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:42.320462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:42.320462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:42.320462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:42.320462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:42.320462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:42.320462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:42.321342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:42.321342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:42.321342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:42.321342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 246/508] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:42.321342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384de145-6a84-416d-b293-ec0a99d350d2 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:42.360536 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 263/509] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Aug 23 06:59:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:42.361641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb629f-8688-48c9-9f1f-25665c6e58ef None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:42.425391 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384de145-6a84-416d-b293-ec0a99d350d2 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:42.461906 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fd47d-25e6-40f1-996a-09899a6a4e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:42.462536 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fd47d-25e6-40f1-996a-09899a6a4e8d None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:42.462808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fd47d-25e6-40f1-996a-09899a6a4e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:42.463071 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fd47d-25e6-40f1-996a-09899a6a4e8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:42.465801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb629f-8688-48c9-9f1f-25665c6e58ef None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:42.474180 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1fd47d-25e6-40f1-996a-09899a6a4e8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:42.475677 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1fd47d-25e6-40f1-996a-09899a6a4e8d None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:42.490117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-da1fd47d-25e6-40f1-996a-09899a6a4e8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:42.490117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:42.490117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:42.490117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:42.490117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:42.490117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:42.490117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:42.490117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:42.490117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:42.490117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:42.490117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:42.490117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:42.490117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:42.490117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:42.490117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:42.490117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:42.491039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:42.491039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:42.491039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:42.491039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:42.491039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:42.491039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:42.491039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:42.491039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:42.491039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:42.491039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:42.491039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:42.491039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:42.491039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:42.491039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:42.491039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:42.491039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:42.491039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:42.491924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:42.491924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:42.491924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:42.491924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 247/510] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:42.492142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fd47d-25e6-40f1-996a-09899a6a4e8d None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:42.503874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea74180-acd4-4549-9d1e-4e4213b8f239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:42.504490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea74180-acd4-4549-9d1e-4e4213b8f239 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:42.504768 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea74180-acd4-4549-9d1e-4e4213b8f239 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:42.505082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea74180-acd4-4549-9d1e-4e4213b8f239 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:42.505728 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea74180-acd4-4549-9d1e-4e4213b8f239 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:42.507190 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea74180-acd4-4549-9d1e-4e4213b8f239 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:42.511243 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:42.528563 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 264/511] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 23 06:59:42 2022] POST /identity/v3/users => generated 247 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:59:42.529370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea74180-acd4-4549-9d1e-4e4213b8f239 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:42.557472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:42.601951 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fd47d-25e6-40f1-996a-09899a6a4e8d None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:42.641870 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae7e9f-66f4-4ebd-9db0-51c0be4a2ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:42.643113 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae7e9f-66f4-4ebd-9db0-51c0be4a2ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:42.643600 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae7e9f-66f4-4ebd-9db0-51c0be4a2ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:42.644008 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae7e9f-66f4-4ebd-9db0-51c0be4a2ce8 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:42.645645 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae7e9f-66f4-4ebd-9db0-51c0be4a2ce8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:42.646129 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae7e9f-66f4-4ebd-9db0-51c0be4a2ce8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:42.648469 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae7e9f-66f4-4ebd-9db0-51c0be4a2ce8 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:42.665917 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 248/512] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 06:59:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:42.665917 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae7e9f-66f4-4ebd-9db0-51c0be4a2ce8 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:42.671482 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea74180-acd4-4549-9d1e-4e4213b8f239 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:42.716605 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397dfb06-5c30-40aa-90e3-1c4d0f2dabec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:42.717512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dfb06-5c30-40aa-90e3-1c4d0f2dabec None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:42.718051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dfb06-5c30-40aa-90e3-1c4d0f2dabec None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:42.718329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dfb06-5c30-40aa-90e3-1c4d0f2dabec None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:42.719000 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397dfb06-5c30-40aa-90e3-1c4d0f2dabec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:42.719493 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397dfb06-5c30-40aa-90e3-1c4d0f2dabec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:42.721381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397dfb06-5c30-40aa-90e3-1c4d0f2dabec None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:42.735223 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 265/513] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:42.736087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397dfb06-5c30-40aa-90e3-1c4d0f2dabec None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:42.760298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:42.783272 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae7e9f-66f4-4ebd-9db0-51c0be4a2ce8 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:42.817185 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:42.824486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40070fb7-17db-4729-88cd-d0f711e3f845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:42.825412 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40070fb7-17db-4729-88cd-d0f711e3f845 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:42.825687 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40070fb7-17db-4729-88cd-d0f711e3f845 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:42.825953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40070fb7-17db-4729-88cd-d0f711e3f845 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:42.856937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397dfb06-5c30-40aa-90e3-1c4d0f2dabec None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:42.894776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72477eb-d5cf-44ad-98f2-27c0d6a996e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:42.895386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72477eb-d5cf-44ad-98f2-27c0d6a996e5 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:42.895694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72477eb-d5cf-44ad-98f2-27c0d6a996e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:42.895968 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72477eb-d5cf-44ad-98f2-27c0d6a996e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:42.896610 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72477eb-d5cf-44ad-98f2-27c0d6a996e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:42.896911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72477eb-d5cf-44ad-98f2-27c0d6a996e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:42.898407 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72477eb-d5cf-44ad-98f2-27c0d6a996e5 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:42.899679 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40070fb7-17db-4729-88cd-d0f711e3f845 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:42.901227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40070fb7-17db-4729-88cd-d0f711e3f845 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:42.914984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 266/514] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:42.914984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72477eb-d5cf-44ad-98f2-27c0d6a996e5 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:42.988993 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:43.015461 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 249/515] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:43.015461 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40070fb7-17db-4729-88cd-d0f711e3f845 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:43.029746 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72477eb-d5cf-44ad-98f2-27c0d6a996e5 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:43.060501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:43.069262 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3dcf6c-95c4-4b59-8782-2942afe7bf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:43.069900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dcf6c-95c4-4b59-8782-2942afe7bf10 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:43.070187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dcf6c-95c4-4b59-8782-2942afe7bf10 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:43.070421 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dcf6c-95c4-4b59-8782-2942afe7bf10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:43.082474 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3dcf6c-95c4-4b59-8782-2942afe7bf10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:43.083963 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3dcf6c-95c4-4b59-8782-2942afe7bf10 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:43.098785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-1b3dcf6c-95c4-4b59-8782-2942afe7bf10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:43.098785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:43.098785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:43.098785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:43.098785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:43.098785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:43.098785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:43.098785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:43.098785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:43.098785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:43.098785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:43.098785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:43.098785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:43.098785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:43.098785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:43.098785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:43.099811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:43.099811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:43.099811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:43.099811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:43.099811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:43.099811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:43.099811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:43.099811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:43.099811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:43.099811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:43.099811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:43.099811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:43.099811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:43.099811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:43.099811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:43.099811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:43.099811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:43.100666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:43.100666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:43.100666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:43.100666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 267/516] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:43.100881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3dcf6c-95c4-4b59-8782-2942afe7bf10 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:43.125657 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40070fb7-17db-4729-88cd-d0f711e3f845 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:43.165521 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafd1a2-ad59-4b3e-837f-6b5cd5240c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:43.166421 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafd1a2-ad59-4b3e-837f-6b5cd5240c13 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:43.166703 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafd1a2-ad59-4b3e-837f-6b5cd5240c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:43.166977 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafd1a2-ad59-4b3e-837f-6b5cd5240c13 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:43.176559 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafd1a2-ad59-4b3e-837f-6b5cd5240c13 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:43.178032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafd1a2-ad59-4b3e-837f-6b5cd5240c13 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:43.194130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-9fafd1a2-ad59-4b3e-837f-6b5cd5240c13 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 06:59:43.194130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:43.194130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:43.194130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:43.194130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:43.194130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:43.194130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:43.194130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:43.194130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:43.194130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:43.194130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:43.194130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:43.194130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:59:43.194130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:59:43.194130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:59:43.194130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:59:43.195104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:43.195104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:43.195104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:59:43.195104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:43.195104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:59:43.195104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:43.195104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:43.195104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:43.195104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:43.195104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:43.195104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:43.195104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:43.195104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:43.195104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:43.195104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:43.195104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:43.195104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:43.195104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:43.196032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:43.196032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:43.196032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:59:43.196032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:59:43.196032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:59:43.196032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:59:43.196032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:59:43.196032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:59:43.196032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 06:59:43.196032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:43.196570 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 250/517] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:43.197158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafd1a2-ad59-4b3e-837f-6b5cd5240c13 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:43.208770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3dcf6c-95c4-4b59-8782-2942afe7bf10 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:43.244060 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:43.246885 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92d61b-3075-47d4-b682-f9ff897e45d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:43.247899 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92d61b-3075-47d4-b682-f9ff897e45d6 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:43.248313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92d61b-3075-47d4-b682-f9ff897e45d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:43.248718 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92d61b-3075-47d4-b682-f9ff897e45d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:43.256011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92d61b-3075-47d4-b682-f9ff897e45d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:43.257619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92d61b-3075-47d4-b682-f9ff897e45d6 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:43.269593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-ee92d61b-3075-47d4-b682-f9ff897e45d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:43.269593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:43.269593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:43.269593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:43.269593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:43.269593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:43.269593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:43.269593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:43.269593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:43.269593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:43.269593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:43.269593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:43.269593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:59:43.269593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:59:43.269593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:59:43.270478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:59:43.270478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:43.270478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:43.270478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:43.270478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:43.270478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:43.270478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:43.270478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:43.270478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:43.270478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:43.270478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:43.270478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:43.270478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:43.270478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:43.270478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:43.270478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:43.270478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:43.271539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:59:43.271539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:59:43.271539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:59:43.271539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:59:43.271539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:59:43.271539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:59:43.271539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:43.271539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:43.273153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 268/518] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:43.274048 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92d61b-3075-47d4-b682-f9ff897e45d6 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:43.311263 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafd1a2-ad59-4b3e-837f-6b5cd5240c13 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:43.321306 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:43.346833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4b830-d979-4466-9f3e-15f508b7d601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:43.347586 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4b830-d979-4466-9f3e-15f508b7d601 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:43.347868 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4b830-d979-4466-9f3e-15f508b7d601 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:43.348139 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4b830-d979-4466-9f3e-15f508b7d601 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:43.358680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4b830-d979-4466-9f3e-15f508b7d601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:43.360186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4b830-d979-4466-9f3e-15f508b7d601 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:43.376085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-e9e4b830-d979-4466-9f3e-15f508b7d601 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:43.376085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:43.376085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:43.376085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:43.376085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:43.376085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:43.376085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:43.376085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:43.376085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:43.376085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:43.376085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:43.376085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:43.376085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:43.376085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:43.376085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:43.376085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:43.377368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:43.377368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:43.377368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:43.377368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:43.377368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:43.377368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:43.377368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:43.377368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:43.377368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:43.377368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:43.377368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:43.377368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:43.377368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:43.377368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:43.377368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:43.377368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:43.377368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:43.378403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:43.378403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:43.378403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:43.381963 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 251/519] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:43.382798 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4b830-d979-4466-9f3e-15f508b7d601 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:43.387350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92d61b-3075-47d4-b682-f9ff897e45d6 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:43.423974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8becee98-d4e5-4b35-ad70-04ac4002053b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:43.424733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becee98-d4e5-4b35-ad70-04ac4002053b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:43.425012 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becee98-d4e5-4b35-ad70-04ac4002053b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:43.425281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becee98-d4e5-4b35-ad70-04ac4002053b None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:43.426147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8becee98-d4e5-4b35-ad70-04ac4002053b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:43.426252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8becee98-d4e5-4b35-ad70-04ac4002053b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:43.427619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8becee98-d4e5-4b35-ad70-04ac4002053b None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:43.444391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 269/520] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:43.444391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8becee98-d4e5-4b35-ad70-04ac4002053b None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:43.482314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:43.492181 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4b830-d979-4466-9f3e-15f508b7d601 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:43.527111 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25ff37-9bf5-4d24-9160-0045d7a015e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:43.531306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25ff37-9bf5-4d24-9160-0045d7a015e1 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:43.531306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25ff37-9bf5-4d24-9160-0045d7a015e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:43.531306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25ff37-9bf5-4d24-9160-0045d7a015e1 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:43.531306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25ff37-9bf5-4d24-9160-0045d7a015e1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:43.531306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25ff37-9bf5-4d24-9160-0045d7a015e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:43.531306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25ff37-9bf5-4d24-9160-0045d7a015e1 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:43.549142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 252/521] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Aug 23 06:59:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:43.549975 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25ff37-9bf5-4d24-9160-0045d7a015e1 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:43.554488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:43.557684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8becee98-d4e5-4b35-ad70-04ac4002053b None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:43.594753 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620f528-db2b-425b-bfe3-685257e383fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:43.595363 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620f528-db2b-425b-bfe3-685257e383fa None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:43.595719 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620f528-db2b-425b-bfe3-685257e383fa None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:43.596000 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620f528-db2b-425b-bfe3-685257e383fa None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:43.596696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620f528-db2b-425b-bfe3-685257e383fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:43.596998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620f528-db2b-425b-bfe3-685257e383fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:43.598450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620f528-db2b-425b-bfe3-685257e383fa None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:43.616042 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 270/522] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 06:59:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:43.616879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620f528-db2b-425b-bfe3-685257e383fa None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:43.666452 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25ff37-9bf5-4d24-9160-0045d7a015e1 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:43.708020 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0a236-a41d-4da9-a7e6-8b353b04d370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:43.708626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0a236-a41d-4da9-a7e6-8b353b04d370 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:43.708903 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0a236-a41d-4da9-a7e6-8b353b04d370 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:43.709400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0a236-a41d-4da9-a7e6-8b353b04d370 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:43.709811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f0a236-a41d-4da9-a7e6-8b353b04d370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:43.710112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f0a236-a41d-4da9-a7e6-8b353b04d370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:43.711509 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f0a236-a41d-4da9-a7e6-8b353b04d370 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:43.723450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 253/523] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:43.724236 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0a236-a41d-4da9-a7e6-8b353b04d370 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:43.732538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620f528-db2b-425b-bfe3-685257e383fa None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:43.741113 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:43.768105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ff09b-8c5a-4498-b74c-06500e62acfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:43.768725 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ff09b-8c5a-4498-b74c-06500e62acfc None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:43.769000 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ff09b-8c5a-4498-b74c-06500e62acfc None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:43.769266 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ff09b-8c5a-4498-b74c-06500e62acfc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:43.776410 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ff09b-8c5a-4498-b74c-06500e62acfc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:43.777866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ff09b-8c5a-4498-b74c-06500e62acfc None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:43.789257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-126ff09b-8c5a-4498-b74c-06500e62acfc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 06:59:43.789257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:43.789257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:43.789257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:43.789257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:43.789257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:43.789257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:43.789257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:43.789257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:43.789257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:43.789257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:43.789257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:43.789257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:59:43.789257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:59:43.789257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:59:43.789257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:59:43.790173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:43.790173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:43.790173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:59:43.790173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:43.790173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:59:43.790173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:43.790173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:43.790173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:43.790173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:43.790173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:43.790173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:43.790173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:43.790173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:43.790173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:43.790173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:43.790173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:43.790173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:43.790173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:43.791201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:43.791201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:43.791201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:59:43.791201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:59:43.791201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:59:43.791201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:59:43.791201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:59:43.791201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:59:43.791201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 06:59:43.791201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:43.791201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 271/524] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 06:59:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:43.791201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ff09b-8c5a-4498-b74c-06500e62acfc None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:43.816418 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:43.837134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0a236-a41d-4da9-a7e6-8b353b04d370 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:43.875310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac30422-a246-4b41-8751-be781a02c044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:43.875948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac30422-a246-4b41-8751-be781a02c044 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:43.876223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac30422-a246-4b41-8751-be781a02c044 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:43.876490 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac30422-a246-4b41-8751-be781a02c044 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:43.907611 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ff09b-8c5a-4498-b74c-06500e62acfc None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:43.950133 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715af7f0-85e9-4604-8fcc-440a3ce1b60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:43.950746 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715af7f0-85e9-4604-8fcc-440a3ce1b60e None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:43.951008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715af7f0-85e9-4604-8fcc-440a3ce1b60e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:43.951279 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715af7f0-85e9-4604-8fcc-440a3ce1b60e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:43.952036 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715af7f0-85e9-4604-8fcc-440a3ce1b60e None admin] RBAC: Adding query filter params (role.id=2d62b8b8395745c7983c6c8d100d985b, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=a8d2cd93042c4583b63afe629d47fce0) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:43.952344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715af7f0-85e9-4604-8fcc-440a3ce1b60e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:43.953804 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715af7f0-85e9-4604-8fcc-440a3ce1b60e None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:43.956872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac30422-a246-4b41-8751-be781a02c044 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:43.958323 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac30422-a246-4b41-8751-be781a02c044 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:43.960080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 272/525] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:59:43 2022] GET /identity/v3/role_assignments?user.id=a8d2cd93042c4583b63afe629d47fce0&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2d62b8b8395745c7983c6c8d100d985b => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:43.961035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715af7f0-85e9-4604-8fcc-440a3ce1b60e None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:43.983323 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:44.080794 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:44.100415 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715af7f0-85e9-4604-8fcc-440a3ce1b60e None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:44.118463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 254/526] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:44.119656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac30422-a246-4b41-8751-be781a02c044 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:44.147380 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348a212-72c6-4b3b-a759-30a73c831955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:44.148143 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348a212-72c6-4b3b-a759-30a73c831955 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:44.148419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348a212-72c6-4b3b-a759-30a73c831955 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:44.148686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348a212-72c6-4b3b-a759-30a73c831955 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:44.158147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348a212-72c6-4b3b-a759-30a73c831955 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:44.160449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348a212-72c6-4b3b-a759-30a73c831955 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:44.175369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-8348a212-72c6-4b3b-a759-30a73c831955 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:44.175369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:44.175369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:44.175369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:44.175369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:44.175369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:44.175369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:44.175369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:44.175369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:44.175369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:44.175369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:44.175369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:44.175369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:59:44.175369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:59:44.175369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:59:44.176402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:59:44.176402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:44.176402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:44.176402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:44.176402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:44.176402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:44.176402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:44.176402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:44.176402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:44.176402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:44.176402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:44.176402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:44.176402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:44.176402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:44.176402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:44.176402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:44.176402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:44.177229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:59:44.177229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:59:44.177229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:59:44.177229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:59:44.177229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:59:44.177229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:59:44.177229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:44.177229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:44.181369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 273/527] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:44.182952 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348a212-72c6-4b3b-a759-30a73c831955 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:44.249244 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac30422-a246-4b41-8751-be781a02c044 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:44.264434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:44.298473 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f114ea-069d-4832-9ff6-9f67c75fcea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:44.300127 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f114ea-069d-4832-9ff6-9f67c75fcea2 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:44.301112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f114ea-069d-4832-9ff6-9f67c75fcea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:44.301592 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f114ea-069d-4832-9ff6-9f67c75fcea2 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:44.302806 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f114ea-069d-4832-9ff6-9f67c75fcea2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:44.303500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f114ea-069d-4832-9ff6-9f67c75fcea2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:44.305787 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f114ea-069d-4832-9ff6-9f67c75fcea2 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:44.313864 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348a212-72c6-4b3b-a759-30a73c831955 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:44.333894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 255/528] 10.222.0.17 () {56 vars in 1139 bytes} [Tue Aug 23 06:59:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:44.333894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f114ea-069d-4832-9ff6-9f67c75fcea2 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:44.363336 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa11d7-18c8-4103-af4b-b37b30bb93dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:44.364489 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa11d7-18c8-4103-af4b-b37b30bb93dd None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:44.364971 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa11d7-18c8-4103-af4b-b37b30bb93dd None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:44.365455 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa11d7-18c8-4103-af4b-b37b30bb93dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:44.377506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa11d7-18c8-4103-af4b-b37b30bb93dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:44.379440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa11d7-18c8-4103-af4b-b37b30bb93dd None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:44.382492 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:44.397197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-f1fa11d7-18c8-4103-af4b-b37b30bb93dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:44.397197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:44.397197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:44.397197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:44.397197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:44.397197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:44.397197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:44.397197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:44.397197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:44.397197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:44.397197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:44.397197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:44.397197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:44.397197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:44.397197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:44.397197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:44.398147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:44.398147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:44.398147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:44.398147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:44.398147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:44.398147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:44.398147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:44.398147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:44.398147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:44.398147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:44.398147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:44.398147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:44.398147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:44.398147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:44.398147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:44.398147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:44.398147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:44.398976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:44.398976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:44.398976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:44.401314 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 274/529] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:44.402481 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa11d7-18c8-4103-af4b-b37b30bb93dd None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:44.441111 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f114ea-069d-4832-9ff6-9f67c75fcea2 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:44.475434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324ae8e-dc20-4254-b44f-447bdbc0dff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:44.476058 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324ae8e-dc20-4254-b44f-447bdbc0dff6 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:44.476333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324ae8e-dc20-4254-b44f-447bdbc0dff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:44.476666 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324ae8e-dc20-4254-b44f-447bdbc0dff6 None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:44.477296 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324ae8e-dc20-4254-b44f-447bdbc0dff6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:44.477597 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324ae8e-dc20-4254-b44f-447bdbc0dff6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:44.479048 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324ae8e-dc20-4254-b44f-447bdbc0dff6 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:44.501914 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 256/530] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 06:59:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:44.503379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324ae8e-dc20-4254-b44f-447bdbc0dff6 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:44.512553 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa11d7-18c8-4103-af4b-b37b30bb93dd None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:44.533355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:44.549833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8f3d4-67f6-46e7-a971-9faa2c58b324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:44.550437 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8f3d4-67f6-46e7-a971-9faa2c58b324 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:44.550715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8f3d4-67f6-46e7-a971-9faa2c58b324 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:44.550978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8f3d4-67f6-46e7-a971-9faa2c58b324 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:44.614362 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:44.614700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324ae8e-dc20-4254-b44f-447bdbc0dff6 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:44.620879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8f3d4-67f6-46e7-a971-9faa2c58b324 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:44.622328 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8f3d4-67f6-46e7-a971-9faa2c58b324 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:44.652962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb577a-34ba-4c8d-abcc-c25c8e8a083d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:44.653643 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb577a-34ba-4c8d-abcc-c25c8e8a083d None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:44.653850 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb577a-34ba-4c8d-abcc-c25c8e8a083d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:44.654125 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb577a-34ba-4c8d-abcc-c25c8e8a083d None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:44.654771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb577a-34ba-4c8d-abcc-c25c8e8a083d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:44.655075 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb577a-34ba-4c8d-abcc-c25c8e8a083d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:44.656508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb577a-34ba-4c8d-abcc-c25c8e8a083d None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:44.669450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 257/531] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:44.672766 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb577a-34ba-4c8d-abcc-c25c8e8a083d None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:44.726843 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 275/532] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:44.732548 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd42de-78f8-4df5-b03a-0f235719af94 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:44.732837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd42de-78f8-4df5-b03a-0f235719af94 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:44.733110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd42de-78f8-4df5-b03a-0f235719af94 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:44.734491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 276/533] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:44.735101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd42de-78f8-4df5-b03a-0f235719af94 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:44.778842 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:44.792215 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb577a-34ba-4c8d-abcc-c25c8e8a083d None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:44.828537 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a2764-0dcf-46ad-9cf9-f10fdb9f67e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:44.829328 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a2764-0dcf-46ad-9cf9-f10fdb9f67e5 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:44.829435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a2764-0dcf-46ad-9cf9-f10fdb9f67e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:44.829659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a2764-0dcf-46ad-9cf9-f10fdb9f67e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:44.830922 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a2764-0dcf-46ad-9cf9-f10fdb9f67e5 None admin] RBAC: Adding query filter params (role.id=2d62b8b8395745c7983c6c8d100d985b, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=57a5f04cb9674b86a6b4b9a472e1afc0) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:44.831244 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a2764-0dcf-46ad-9cf9-f10fdb9f67e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:44.832828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a2764-0dcf-46ad-9cf9-f10fdb9f67e5 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:44.847463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 258/534] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:59:44 2022] GET /identity/v3/role_assignments?user.id=57a5f04cb9674b86a6b4b9a472e1afc0&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2d62b8b8395745c7983c6c8d100d985b => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:44.847463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a2764-0dcf-46ad-9cf9-f10fdb9f67e5 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:44.854226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:44.864595 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd42de-78f8-4df5-b03a-0f235719af94 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:44.910511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781ec4a-b7c2-4e42-8da2-571a53990335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:44.911263 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781ec4a-b7c2-4e42-8da2-571a53990335 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:44.911811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781ec4a-b7c2-4e42-8da2-571a53990335 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:44.912156 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781ec4a-b7c2-4e42-8da2-571a53990335 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:44.922059 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781ec4a-b7c2-4e42-8da2-571a53990335 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:44.924440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781ec4a-b7c2-4e42-8da2-571a53990335 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:44.939972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-a781ec4a-b7c2-4e42-8da2-571a53990335 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:44.939972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:44.939972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:44.939972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:44.939972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:44.939972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:44.939972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:44.939972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:44.939972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:44.939972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:44.939972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:44.939972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:44.939972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:59:44.939972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:59:44.939972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:59:44.941004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:59:44.941004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:44.941004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:44.941004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:44.941004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:44.941004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:44.941004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:44.941004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:44.941004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:44.941004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:44.941004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:44.941004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:44.941004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:44.941004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:44.941004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:44.941004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:44.941004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:44.942031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:59:44.942031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:59:44.942031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:59:44.942031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:59:44.942031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:59:44.942031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:59:44.942031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:44.942031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:44.943056 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 277/535] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:44.944431 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b970aa-987f-4082-8200-35f85edb9708 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:44.944720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b970aa-987f-4082-8200-35f85edb9708 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:44.944987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b970aa-987f-4082-8200-35f85edb9708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:44.968713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-33b970aa-987f-4082-8200-35f85edb9708 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:44.970454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-33b970aa-987f-4082-8200-35f85edb9708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:44.972261 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a2764-0dcf-46ad-9cf9-f10fdb9f67e5 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:45.032603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b8aba-f76f-4d20-b4a9-b4d40524dc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:45.033566 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b8aba-f76f-4d20-b4a9-b4d40524dc06 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:45.034006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b8aba-f76f-4d20-b4a9-b4d40524dc06 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:45.034486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b8aba-f76f-4d20-b4a9-b4d40524dc06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:45.047519 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b8aba-f76f-4d20-b4a9-b4d40524dc06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:45.049542 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b8aba-f76f-4d20-b4a9-b4d40524dc06 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:45.066068 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 278/536] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:45.067006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-8e4b8aba-f76f-4d20-b4a9-b4d40524dc06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:45.067006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:45.067006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:45.067006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:45.067006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:45.067006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:45.067006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:45.067006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:45.067006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:45.067006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:45.067006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:45.067006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:45.067006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:45.067006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:45.067006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:45.067006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:45.068034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:45.068034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:45.068034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:45.068034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:45.068034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:45.068034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:45.068034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:45.068034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:45.068034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:45.068034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:45.068034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:45.068034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:45.068034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:45.068034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:45.068034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:45.068034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:45.068034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:45.068863 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:45.068863 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:45.068863 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:45.069023 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b970aa-987f-4082-8200-35f85edb9708 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:45.075757 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 259/537] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:45.077251 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b8aba-f76f-4d20-b4a9-b4d40524dc06 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:45.087738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:45.222646 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b8aba-f76f-4d20-b4a9-b4d40524dc06 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:45.225253 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:45.258895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b970aa-987f-4082-8200-35f85edb9708 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:45.277525 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abdea26-b889-46eb-bf77-c4e6f727acfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:45.278482 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abdea26-b889-46eb-bf77-c4e6f727acfe None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:45.278961 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abdea26-b889-46eb-bf77-c4e6f727acfe None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:45.279444 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abdea26-b889-46eb-bf77-c4e6f727acfe None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:45.280337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abdea26-b889-46eb-bf77-c4e6f727acfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:45.282389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abdea26-b889-46eb-bf77-c4e6f727acfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:45.284238 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abdea26-b889-46eb-bf77-c4e6f727acfe None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:45.315485 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 260/538] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:45.317244 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abdea26-b889-46eb-bf77-c4e6f727acfe None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:45.352717 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab27695-2dd4-4eab-b3e5-dcd02334b35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:45.353346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab27695-2dd4-4eab-b3e5-dcd02334b35c None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:45.353626 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab27695-2dd4-4eab-b3e5-dcd02334b35c None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:45.353906 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab27695-2dd4-4eab-b3e5-dcd02334b35c None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:45.354616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab27695-2dd4-4eab-b3e5-dcd02334b35c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:45.354924 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab27695-2dd4-4eab-b3e5-dcd02334b35c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:45.362666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab27695-2dd4-4eab-b3e5-dcd02334b35c None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:45.379555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 279/539] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 06:59:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:45.384521 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab27695-2dd4-4eab-b3e5-dcd02334b35c None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:45.406890 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:45.449941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abdea26-b889-46eb-bf77-c4e6f727acfe None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:45.489786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17492f87-b73a-4bc1-ad30-d7329469205a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:45.490339 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17492f87-b73a-4bc1-ad30-d7329469205a None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:45.490616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17492f87-b73a-4bc1-ad30-d7329469205a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:45.490886 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17492f87-b73a-4bc1-ad30-d7329469205a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:45.497871 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17492f87-b73a-4bc1-ad30-d7329469205a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:45.499395 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17492f87-b73a-4bc1-ad30-d7329469205a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:45.507454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab27695-2dd4-4eab-b3e5-dcd02334b35c None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:45.512314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-17492f87-b73a-4bc1-ad30-d7329469205a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:45.512314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:45.512314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:45.512314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:45.512314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:45.512314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:45.512314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:45.512314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:45.512314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:45.512314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:45.512314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:45.512314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:45.512314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:59:45.512314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:59:45.512314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:59:45.513137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:59:45.513137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:45.513137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:45.513137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:45.513137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:45.513137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:45.513137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:45.513137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:45.513137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:45.513137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:45.513137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:45.513137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:45.513137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:45.513137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:45.513137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:45.513137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:45.513137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:45.518908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:59:45.518908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:59:45.518908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:59:45.518908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:59:45.518908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:59:45.518908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:59:45.518908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:45.518908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:45.518908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 261/540] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:45.518908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17492f87-b73a-4bc1-ad30-d7329469205a None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:45.549800 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef72ad-5349-42ae-a177-2a50667db8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:45.550505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef72ad-5349-42ae-a177-2a50667db8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:45.550784 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef72ad-5349-42ae-a177-2a50667db8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:45.551054 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef72ad-5349-42ae-a177-2a50667db8a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:45.551936 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef72ad-5349-42ae-a177-2a50667db8a7 None admin] RBAC: Adding query filter params (role.id=2d62b8b8395745c7983c6c8d100d985b, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=a26f09e892a74d9eab97e73cce9d062a) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:45.552240 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef72ad-5349-42ae-a177-2a50667db8a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:45.553725 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef72ad-5349-42ae-a177-2a50667db8a7 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:45.559731 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 280/541] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:59:45 2022] GET /identity/v3/role_assignments?user.id=a26f09e892a74d9eab97e73cce9d062a&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2d62b8b8395745c7983c6c8d100d985b => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:45.560394 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef72ad-5349-42ae-a177-2a50667db8a7 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:45.575777 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:45.697157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17492f87-b73a-4bc1-ad30-d7329469205a None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:45.751879 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:45.764341 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef72ad-5349-42ae-a177-2a50667db8a7 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:45.776775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baafb14-3c64-48db-96d5-2e0bfb2c4f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:45.777425 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baafb14-3c64-48db-96d5-2e0bfb2c4f15 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:45.777707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baafb14-3c64-48db-96d5-2e0bfb2c4f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:45.777978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baafb14-3c64-48db-96d5-2e0bfb2c4f15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:45.797709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baafb14-3c64-48db-96d5-2e0bfb2c4f15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:45.803229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-3baafb14-3c64-48db-96d5-2e0bfb2c4f15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:45.803229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:45.803229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:45.803229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:45.803229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:45.803229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:45.803229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:45.803229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:45.803229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:45.803229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:45.803229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:45.803229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:45.803229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:45.803229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:45.803229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:45.803229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:45.804117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:45.804117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:45.804117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:45.804117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:45.804117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:45.804117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:45.804117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:45.804117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:45.804117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:45.804117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:45.804117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:45.804117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:45.804117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:45.804117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:45.804117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:45.804117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:45.804891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:45.804891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:45.804891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:45.804891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:45.804891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:45.804891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:45.804891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:45.804891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:45.804891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:45.804891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:45.804891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:45.804891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:45.806089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 262/542] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 06:59:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:45.811299 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baafb14-3c64-48db-96d5-2e0bfb2c4f15 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:45.836238 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec77bb-915f-46f3-86db-7b1ffffd7931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:45.836918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec77bb-915f-46f3-86db-7b1ffffd7931 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:45.837194 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec77bb-915f-46f3-86db-7b1ffffd7931 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:45.837466 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec77bb-915f-46f3-86db-7b1ffffd7931 None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:45.838169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec77bb-915f-46f3-86db-7b1ffffd7931 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:45.838469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec77bb-915f-46f3-86db-7b1ffffd7931 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:45.839945 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec77bb-915f-46f3-86db-7b1ffffd7931 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:45.858011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 281/543] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 06:59:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:45.865654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec77bb-915f-46f3-86db-7b1ffffd7931 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:45.973953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:45.981027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baafb14-3c64-48db-96d5-2e0bfb2c4f15 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:46.013448 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec77bb-915f-46f3-86db-7b1ffffd7931 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:46.029837 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2710edf-ba6c-46ed-97fa-f11fcd6ecbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:46.030627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2710edf-ba6c-46ed-97fa-f11fcd6ecbfa None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:46.031163 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2710edf-ba6c-46ed-97fa-f11fcd6ecbfa None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:46.031574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2710edf-ba6c-46ed-97fa-f11fcd6ecbfa None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:46.032733 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2710edf-ba6c-46ed-97fa-f11fcd6ecbfa None admin] RBAC: Adding query filter params (name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:46.033143 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2710edf-ba6c-46ed-97fa-f11fcd6ecbfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:46.034968 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2710edf-ba6c-46ed-97fa-f11fcd6ecbfa None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:46.047030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 263/544] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 06:59:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:46.048895 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249f9e7-b9f3-4218-a318-bd46d2fcd4ba None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:46.049481 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249f9e7-b9f3-4218-a318-bd46d2fcd4ba None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:46.049856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249f9e7-b9f3-4218-a318-bd46d2fcd4ba None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:46.051790 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 264/545] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:46.053088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249f9e7-b9f3-4218-a318-bd46d2fcd4ba None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:46.058448 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0dd583-f560-4848-8f81-648e82375ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:46.059262 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0dd583-f560-4848-8f81-648e82375ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:46.059691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0dd583-f560-4848-8f81-648e82375ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:46.060090 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0dd583-f560-4848-8f81-648e82375ac2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:46.061021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0dd583-f560-4848-8f81-648e82375ac2 None admin] RBAC: Adding query filter params (role.id=2d62b8b8395745c7983c6c8d100d985b, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=938364c02d8d4dd2ae832c5129be3cbe) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:46.061576 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0dd583-f560-4848-8f81-648e82375ac2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:46.063802 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0dd583-f560-4848-8f81-648e82375ac2 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:46.075599 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 282/546] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:59:45 2022] GET /identity/v3/role_assignments?user.id=938364c02d8d4dd2ae832c5129be3cbe&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2d62b8b8395745c7983c6c8d100d985b => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:46.079440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ddd61-1f7e-4629-808e-032252920927 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:46.079440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ddd61-1f7e-4629-808e-032252920927 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:46.079440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ddd61-1f7e-4629-808e-032252920927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:46.116125 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-e10ddd61-1f7e-4629-808e-032252920927 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:46.117351 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e10ddd61-1f7e-4629-808e-032252920927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:46.147765 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:46.255863 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249f9e7-b9f3-4218-a318-bd46d2fcd4ba None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:46.256288 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 283/547] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:46.297002 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df302c2a-a1df-4c5d-8b0e-9627aa750633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:46.297874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df302c2a-a1df-4c5d-8b0e-9627aa750633 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:46.298290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df302c2a-a1df-4c5d-8b0e-9627aa750633 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:46.298700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df302c2a-a1df-4c5d-8b0e-9627aa750633 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:46.309593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df302c2a-a1df-4c5d-8b0e-9627aa750633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:46.311221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df302c2a-a1df-4c5d-8b0e-9627aa750633 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:46.325793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-df302c2a-a1df-4c5d-8b0e-9627aa750633 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:46.325793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:46.325793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:46.325793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:46.325793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:46.325793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:46.325793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:46.325793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:46.325793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:46.325793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:46.325793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:46.325793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:46.325793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:46.325793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:46.325793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:46.325793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:46.326872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:46.326872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:46.326872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:46.326872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:46.326872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:46.326872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:46.326872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:46.326872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:46.326872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:46.326872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:46.326872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:46.326872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:46.326872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:46.326872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:46.326872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:46.326872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:46.326872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:46.327962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:46.327962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:46.327962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:46.329627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 265/548] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:46.335089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ddd61-1f7e-4629-808e-032252920927 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:46.403872 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:46.458873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ddd61-1f7e-4629-808e-032252920927 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:46.514865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df42063-7b30-4e27-af87-06e451ef5c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:46.515536 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df42063-7b30-4e27-af87-06e451ef5c79 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:46.515823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df42063-7b30-4e27-af87-06e451ef5c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:46.516096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df42063-7b30-4e27-af87-06e451ef5c79 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:46.516842 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df42063-7b30-4e27-af87-06e451ef5c79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:46.517150 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df42063-7b30-4e27-af87-06e451ef5c79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:46.518701 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df42063-7b30-4e27-af87-06e451ef5c79 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:46.534616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 284/549] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:46.539830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df302c2a-a1df-4c5d-8b0e-9627aa750633 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:46.590543 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df42063-7b30-4e27-af87-06e451ef5c79 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:46.664574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:46.669347 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df302c2a-a1df-4c5d-8b0e-9627aa750633 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:46.704404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b83ae-3824-48c4-b3c2-8b32ea90f455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:46.705469 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b83ae-3824-48c4-b3c2-8b32ea90f455 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:46.705939 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b83ae-3824-48c4-b3c2-8b32ea90f455 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:46.706399 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b83ae-3824-48c4-b3c2-8b32ea90f455 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:46.715545 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b83ae-3824-48c4-b3c2-8b32ea90f455 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:46.717339 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b83ae-3824-48c4-b3c2-8b32ea90f455 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:46.729471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-c96b83ae-3824-48c4-b3c2-8b32ea90f455 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 06:59:46.729471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:46.729471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:46.729471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:46.729471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:46.729471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:46.729471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:46.729471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:46.729471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:46.729471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:46.729471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:46.729471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:46.729471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:59:46.729471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:59:46.729471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:59:46.729471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:59:46.730947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:46.730947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:46.730947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:59:46.730947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:46.730947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:59:46.730947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:46.730947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:46.730947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:46.730947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:46.730947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:46.730947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:46.730947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:46.730947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:46.730947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:46.730947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:46.730947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:46.730947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:46.730947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:46.732027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:46.732027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:46.732027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:59:46.732027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:59:46.732027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:59:46.732027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:59:46.732027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:59:46.732027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:59:46.732027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 06:59:46.732027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:46.734560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 266/550] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 06:59:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:46.739730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b83ae-3824-48c4-b3c2-8b32ea90f455 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:46.813850 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df42063-7b30-4e27-af87-06e451ef5c79 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:46.852848 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b83ae-3824-48c4-b3c2-8b32ea90f455 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:46.871434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:46.884036 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f68037-74e1-44aa-be19-eaaccc4b4a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:46.884687 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f68037-74e1-44aa-be19-eaaccc4b4a48 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:46.884964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f68037-74e1-44aa-be19-eaaccc4b4a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:46.885243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f68037-74e1-44aa-be19-eaaccc4b4a48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:46.907381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f68037-74e1-44aa-be19-eaaccc4b4a48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:46.915379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-d4f68037-74e1-44aa-be19-eaaccc4b4a48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:46.915379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:46.915379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:46.915379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:46.915379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:46.915379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:46.915379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:46.915379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:46.915379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:46.915379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:46.915379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:46.915379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:46.915379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:46.915379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:46.915379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:46.915379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:46.916350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:46.916350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:46.916350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:46.916350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:46.916350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:46.916350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:46.916350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:46.916350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:46.916350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:46.916350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:46.916350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:46.916350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:46.916350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:46.916350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:46.916350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:46.916350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:46.917765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:46.917765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:46.917765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:46.917765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:46.917765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:46.917765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:46.917765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:46.917765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:46.917765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:46.917765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:46.917765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:46.917765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:46.917765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 285/551] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 06:59:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:46.925920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5b71a-4f1f-4842-b0cf-87c3853ad490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:46.926535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5b71a-4f1f-4842-b0cf-87c3853ad490 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:46.926813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5b71a-4f1f-4842-b0cf-87c3853ad490 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:46.927084 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5b71a-4f1f-4842-b0cf-87c3853ad490 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:46.927909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5b71a-4f1f-4842-b0cf-87c3853ad490 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:46.928211 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5b71a-4f1f-4842-b0cf-87c3853ad490 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:46.929666 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5b71a-4f1f-4842-b0cf-87c3853ad490 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:46.930395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f68037-74e1-44aa-be19-eaaccc4b4a48 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:46.961004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 267/552] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Aug 23 06:59:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:46.968483 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5b71a-4f1f-4842-b0cf-87c3853ad490 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:47.005511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:47.109878 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f68037-74e1-44aa-be19-eaaccc4b4a48 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:47.147823 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5b71a-4f1f-4842-b0cf-87c3853ad490 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:47.167649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cf841c-eed0-45c1-a46c-4299aaebfc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:47.168264 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf841c-eed0-45c1-a46c-4299aaebfc6a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:47.168543 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf841c-eed0-45c1-a46c-4299aaebfc6a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:47.168817 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf841c-eed0-45c1-a46c-4299aaebfc6a None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:47.169611 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cf841c-eed0-45c1-a46c-4299aaebfc6a None admin] RBAC: Adding query filter params (name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:47.169916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cf841c-eed0-45c1-a46c-4299aaebfc6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:47.171303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cf841c-eed0-45c1-a46c-4299aaebfc6a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:47.181164 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 286/553] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 06:59:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:47.182539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4766d0d-28d1-4734-8ea1-5d02760c6e7d None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:47.182793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4766d0d-28d1-4734-8ea1-5d02760c6e7d None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:47.183068 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4766d0d-28d1-4734-8ea1-5d02760c6e7d None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:47.188498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 287/554] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:47.189297 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4766d0d-28d1-4734-8ea1-5d02760c6e7d None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:47.189909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2233152-337e-4282-9ee6-dc8bfa8b5001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:47.190655 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2233152-337e-4282-9ee6-dc8bfa8b5001 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:47.190931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2233152-337e-4282-9ee6-dc8bfa8b5001 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:47.191199 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2233152-337e-4282-9ee6-dc8bfa8b5001 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:47.247352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:47.269883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2233152-337e-4282-9ee6-dc8bfa8b5001 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:47.271722 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2233152-337e-4282-9ee6-dc8bfa8b5001 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:47.290281 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:47.308371 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4766d0d-28d1-4734-8ea1-5d02760c6e7d None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:47.345959 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f301f-cdff-4465-9c24-c070c80f24a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:47.346603 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f301f-cdff-4465-9c24-c070c80f24a5 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:47.346875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f301f-cdff-4465-9c24-c070c80f24a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:47.347259 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f301f-cdff-4465-9c24-c070c80f24a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:47.358371 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f301f-cdff-4465-9c24-c070c80f24a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:47.360032 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f301f-cdff-4465-9c24-c070c80f24a5 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:47.375283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-4e0f301f-cdff-4465-9c24-c070c80f24a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:47.375283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:47.375283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:47.375283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:47.375283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:47.375283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:47.375283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:47.375283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:47.375283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:47.375283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:47.375283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:47.375283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:47.375283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:47.375283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:47.375283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:47.375283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:47.376261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:47.376261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:47.376261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:47.376261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:47.376261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:47.376261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:47.376261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:47.376261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:47.376261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:47.376261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:47.376261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:47.376261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:47.376261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:47.376261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:47.376261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:47.376261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:47.376261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:47.377770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:47.377770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:47.377770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:47.377770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 288/555] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:47.378136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa495411-0f30-4a4a-a08a-51aeea5a3154 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:47.378368 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa495411-0f30-4a4a-a08a-51aeea5a3154 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:47.378675 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa495411-0f30-4a4a-a08a-51aeea5a3154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:47.387058 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 268/556] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:47.388331 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2233152-337e-4282-9ee6-dc8bfa8b5001 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:47.397686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-aa495411-0f30-4a4a-a08a-51aeea5a3154 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:47.398877 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-aa495411-0f30-4a4a-a08a-51aeea5a3154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:47.475546 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 289/557] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:47.476532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa495411-0f30-4a4a-a08a-51aeea5a3154 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:47.494024 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2233152-337e-4282-9ee6-dc8bfa8b5001 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:47.513938 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:47.524281 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:47.560907 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f80f52-d60e-4fe9-bdf0-fe838f466c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:47.561451 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f80f52-d60e-4fe9-bdf0-fe838f466c7d None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:47.561820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f80f52-d60e-4fe9-bdf0-fe838f466c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:47.562218 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f80f52-d60e-4fe9-bdf0-fe838f466c7d None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:47.562995 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f80f52-d60e-4fe9-bdf0-fe838f466c7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:47.563393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f80f52-d60e-4fe9-bdf0-fe838f466c7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:47.565011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f80f52-d60e-4fe9-bdf0-fe838f466c7d None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:47.585578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 269/558] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:47.592586 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f80f52-d60e-4fe9-bdf0-fe838f466c7d None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:47.593840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa495411-0f30-4a4a-a08a-51aeea5a3154 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:47.632044 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877590d-db98-4584-8cda-4ef40c041815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:47.632946 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877590d-db98-4584-8cda-4ef40c041815 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:47.633487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877590d-db98-4584-8cda-4ef40c041815 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:47.633973 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877590d-db98-4584-8cda-4ef40c041815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:47.645389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877590d-db98-4584-8cda-4ef40c041815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:47.647127 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877590d-db98-4584-8cda-4ef40c041815 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:47.663509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-c877590d-db98-4584-8cda-4ef40c041815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:47.663509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:47.663509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:47.663509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:47.663509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:47.663509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:47.663509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:47.663509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:47.663509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:47.663509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:47.663509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:47.663509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:47.663509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:47.663509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:47.663509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:47.663509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:47.664954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:47.664954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:47.664954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:47.664954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:47.664954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:47.664954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:47.664954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:47.664954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:47.664954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:47.664954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:47.664954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:47.664954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:47.664954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:47.664954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:47.664954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:47.664954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:47.664954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:47.665846 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:47.665846 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:47.665846 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:47.669859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 290/559] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:47.673629 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877590d-db98-4584-8cda-4ef40c041815 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:47.787377 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f80f52-d60e-4fe9-bdf0-fe838f466c7d None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:47.808558 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:47.831106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658556f-4782-4008-bc63-a9daa614b84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:47.831729 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877590d-db98-4584-8cda-4ef40c041815 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:47.831955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658556f-4782-4008-bc63-a9daa614b84a None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:47.832235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658556f-4782-4008-bc63-a9daa614b84a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:47.832503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658556f-4782-4008-bc63-a9daa614b84a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:47.846168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1658556f-4782-4008-bc63-a9daa614b84a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:47.848654 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1658556f-4782-4008-bc63-a9daa614b84a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:47.864189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-1658556f-4782-4008-bc63-a9daa614b84a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:59:47.864189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:47.864189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:47.864189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:47.864189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:47.864189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:47.864189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:47.864189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:47.864189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:47.864189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:47.864189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:47.864189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:47.864189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:59:47.864189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:59:47.864189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:59:47.864189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:59:47.865187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:47.865187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:47.865187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:59:47.865187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:47.865187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:59:47.865187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:47.865187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:47.865187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:47.865187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:47.865187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:47.865187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:47.865187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:47.865187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:47.865187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:47.865187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:47.865187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:47.865187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:47.865187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:47.866228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:47.866228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:47.866228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:59:47.866228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:59:47.866228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:59:47.866228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:59:47.866228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:59:47.866228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:59:47.866228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:59:47.866228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:47.867568 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 270/560] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Aug 23 06:59:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:47.868816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb1304-fdd3-40a8-9666-af51cdb5fbab None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:47.869100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb1304-fdd3-40a8-9666-af51cdb5fbab None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:47.869374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb1304-fdd3-40a8-9666-af51cdb5fbab None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:47.871036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 271/561] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:47.871760 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb1304-fdd3-40a8-9666-af51cdb5fbab None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:47.878732 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ecea8-7cbc-4cbf-9e63-8e9efc45bf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:47.879332 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ecea8-7cbc-4cbf-9e63-8e9efc45bf42 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:47.879644 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ecea8-7cbc-4cbf-9e63-8e9efc45bf42 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:47.879919 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ecea8-7cbc-4cbf-9e63-8e9efc45bf42 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:47.880567 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ecea8-7cbc-4cbf-9e63-8e9efc45bf42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:47.880863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ecea8-7cbc-4cbf-9e63-8e9efc45bf42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:47.882300 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ecea8-7cbc-4cbf-9e63-8e9efc45bf42 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:47.895912 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 291/562] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:47.900522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ecea8-7cbc-4cbf-9e63-8e9efc45bf42 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:47.910840 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:47.987516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb1304-fdd3-40a8-9666-af51cdb5fbab None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:48.013069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ecea8-7cbc-4cbf-9e63-8e9efc45bf42 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:48.024726 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c40af-f969-4965-b585-87baa2e43fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:48.025952 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c40af-f969-4965-b585-87baa2e43fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:48.026435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c40af-f969-4965-b585-87baa2e43fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:48.026882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c40af-f969-4965-b585-87baa2e43fe9 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:48.027841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c40af-f969-4965-b585-87baa2e43fe9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:48.028146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c40af-f969-4965-b585-87baa2e43fe9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:48.029598 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c40af-f969-4965-b585-87baa2e43fe9 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:48.052315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 272/563] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Aug 23 06:59:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:48.052315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63190878-49a0-496c-8a55-c6aaae9ba7c1 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:48.052315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63190878-49a0-496c-8a55-c6aaae9ba7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:48.052315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63190878-49a0-496c-8a55-c6aaae9ba7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:48.054727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431da1f-97cc-4369-ac07-e3b6cb1617fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:48.055391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431da1f-97cc-4369-ac07-e3b6cb1617fd None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:48.055699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431da1f-97cc-4369-ac07-e3b6cb1617fd None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:48.055948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431da1f-97cc-4369-ac07-e3b6cb1617fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:48.064596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:48.067902 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431da1f-97cc-4369-ac07-e3b6cb1617fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:48.070918 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-63190878-49a0-496c-8a55-c6aaae9ba7c1 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:48.072333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-63190878-49a0-496c-8a55-c6aaae9ba7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:48.073021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-7431da1f-97cc-4369-ac07-e3b6cb1617fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:48.073021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:48.073021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:48.073021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:48.073021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:48.073021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:48.073021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:48.073021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:48.073021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:48.073021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:48.073021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:48.073021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:48.073021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:48.073021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:48.073021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:48.073021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:48.073932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:48.073932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:48.073932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:48.073932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:48.073932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:48.073932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:48.073932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:48.073932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:48.073932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:48.073932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:48.073932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:48.073932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:48.073932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:48.073932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:48.073932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:48.073932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:48.074763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:48.074763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:48.074763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:48.074763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:48.074763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:48.074763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:48.074763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:48.074763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:48.074763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:48.074763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:48.074763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:48.074763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:48.074763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 292/564] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 06:59:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:48.075810 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431da1f-97cc-4369-ac07-e3b6cb1617fd None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:48.155552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 273/565] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:48.155552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63190878-49a0-496c-8a55-c6aaae9ba7c1 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:48.184709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:48.191881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431da1f-97cc-4369-ac07-e3b6cb1617fd None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:48.229807 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4c590-5e93-4280-96c4-438015b14172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:48.230439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4c590-5e93-4280-96c4-438015b14172 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:48.230718 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4c590-5e93-4280-96c4-438015b14172 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:48.231009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4c590-5e93-4280-96c4-438015b14172 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:48.238132 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4c590-5e93-4280-96c4-438015b14172 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:48.239633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4c590-5e93-4280-96c4-438015b14172 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:48.251142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-87e4c590-5e93-4280-96c4-438015b14172 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:48.251142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:48.251142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:48.251142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:48.251142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:48.251142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:48.251142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:48.251142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:48.251142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:48.251142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:48.251142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:48.251142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:48.251142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:59:48.251142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:59:48.251142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:59:48.252051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:59:48.252051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:48.252051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:48.252051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:48.252051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:48.252051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:48.252051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:48.252051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:48.252051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:48.252051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:48.252051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:48.252051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:48.252051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:48.252051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:48.252051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:48.252051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:48.252051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:48.253113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:59:48.253113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:59:48.253113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:59:48.253113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:59:48.253113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:59:48.253113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:59:48.253113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:48.253113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:48.253113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 293/566] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:48.253113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4c590-5e93-4280-96c4-438015b14172 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:48.272484 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63190878-49a0-496c-8a55-c6aaae9ba7c1 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:48.308443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6cd15-6f16-4274-a9a0-c12a52f51785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:48.309079 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6cd15-6f16-4274-a9a0-c12a52f51785 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:48.309361 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6cd15-6f16-4274-a9a0-c12a52f51785 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:48.309635 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6cd15-6f16-4274-a9a0-c12a52f51785 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:48.313921 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:48.320502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6cd15-6f16-4274-a9a0-c12a52f51785 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:48.322008 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6cd15-6f16-4274-a9a0-c12a52f51785 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:48.337357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-27f6cd15-6f16-4274-a9a0-c12a52f51785 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:48.337357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:48.337357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:48.337357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:48.337357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:48.337357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:48.337357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:48.337357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:48.337357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:48.337357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:48.337357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:48.337357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:48.337357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:48.337357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:48.337357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:48.337357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:48.338311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:48.338311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:48.338311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:48.338311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:48.338311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:48.338311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:48.338311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:48.338311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:48.338311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:48.338311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:48.338311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:48.338311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:48.338311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:48.338311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:48.338311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:48.338311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:48.338311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:48.339197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:48.339197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:48.339197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:48.339197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 274/567] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:48.339870 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313c751-f5bc-4147-833e-26659b90ca80 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:48.339983 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313c751-f5bc-4147-833e-26659b90ca80 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:48.340254 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313c751-f5bc-4147-833e-26659b90ca80 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:48.345503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 275/568] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:48.345503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313c751-f5bc-4147-833e-26659b90ca80 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:48.365596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4c590-5e93-4280-96c4-438015b14172 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:48.403084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394ee33-f676-4464-83df-fa237c4991f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:48.403861 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394ee33-f676-4464-83df-fa237c4991f2 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:48.404145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394ee33-f676-4464-83df-fa237c4991f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:48.404416 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394ee33-f676-4464-83df-fa237c4991f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:48.405072 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394ee33-f676-4464-83df-fa237c4991f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:48.405373 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394ee33-f676-4464-83df-fa237c4991f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:48.407045 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394ee33-f676-4464-83df-fa237c4991f2 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:48.419803 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 294/569] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 06:59:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:48.419803 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394ee33-f676-4464-83df-fa237c4991f2 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:48.422957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:48.449310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313c751-f5bc-4147-833e-26659b90ca80 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:48.484238 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05864f5c-1108-4d31-939d-a35695cbf1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:48.484848 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05864f5c-1108-4d31-939d-a35695cbf1b4 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:48.485128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05864f5c-1108-4d31-939d-a35695cbf1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:48.485401 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05864f5c-1108-4d31-939d-a35695cbf1b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:48.486103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05864f5c-1108-4d31-939d-a35695cbf1b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:48.486408 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05864f5c-1108-4d31-939d-a35695cbf1b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:48.488110 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05864f5c-1108-4d31-939d-a35695cbf1b4 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:48.499818 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 276/570] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 06:59:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:48.501036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d34614-4d64-45f8-b2eb-616a0d6374ef None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:48.501324 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d34614-4d64-45f8-b2eb-616a0d6374ef None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:48.501591 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d34614-4d64-45f8-b2eb-616a0d6374ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:48.519099 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-23d34614-4d64-45f8-b2eb-616a0d6374ef None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:48.519263 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394ee33-f676-4464-83df-fa237c4991f2 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:48.520319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-23d34614-4d64-45f8-b2eb-616a0d6374ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:48.541281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:48.553321 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c0e4b-4997-41b8-b5a7-c467683cf4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:48.553933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c0e4b-4997-41b8-b5a7-c467683cf4cb None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:48.554214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c0e4b-4997-41b8-b5a7-c467683cf4cb None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:48.554490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c0e4b-4997-41b8-b5a7-c467683cf4cb None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:48.555134 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c0e4b-4997-41b8-b5a7-c467683cf4cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:48.555463 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c0e4b-4997-41b8-b5a7-c467683cf4cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:48.556847 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c0e4b-4997-41b8-b5a7-c467683cf4cb None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:48.568610 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 295/571] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:48.569418 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c0e4b-4997-41b8-b5a7-c467683cf4cb None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:48.587833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 277/572] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:48.588680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d34614-4d64-45f8-b2eb-616a0d6374ef None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:48.661761 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:48.690529 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c0e4b-4997-41b8-b5a7-c467683cf4cb None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:48.701160 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d34614-4d64-45f8-b2eb-616a0d6374ef None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:48.729635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea91796-3379-4ab3-ac26-3d00ab1fb315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:48.730293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea91796-3379-4ab3-ac26-3d00ab1fb315 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:48.730651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea91796-3379-4ab3-ac26-3d00ab1fb315 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:48.730836 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea91796-3379-4ab3-ac26-3d00ab1fb315 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:48.736218 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7f7b2-4d84-4e88-bbc8-467819696140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:48.737026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7f7b2-4d84-4e88-bbc8-467819696140 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:48.737396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7f7b2-4d84-4e88-bbc8-467819696140 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:48.737763 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7f7b2-4d84-4e88-bbc8-467819696140 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:48.741631 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea91796-3379-4ab3-ac26-3d00ab1fb315 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:48.746435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-aea91796-3379-4ab3-ac26-3d00ab1fb315 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:48.746435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:48.746435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:48.746435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:48.746435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:48.746435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:48.746435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:48.746435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:48.746435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:48.746435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:48.746435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:48.746435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:48.746435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:48.746435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:48.746435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:48.746435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:48.747550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:48.747550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:48.747550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:48.747550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:48.747550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:48.747550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:48.747550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:48.747550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:48.747550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:48.747550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:48.747550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:48.747550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:48.747550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:48.747550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:48.747550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:48.747550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:48.752633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:48.752633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:48.752633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:48.752633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:48.752633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:48.752633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:48.752633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:48.752633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:48.752633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:48.752633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:48.752633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:48.752633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:48.752633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 296/573] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 06:59:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:48.752633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea91796-3379-4ab3-ac26-3d00ab1fb315 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:48.753550 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7f7b2-4d84-4e88-bbc8-467819696140 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:48.754631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7f7b2-4d84-4e88-bbc8-467819696140 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:48.769305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-f2f7f7b2-4d84-4e88-bbc8-467819696140 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:48.769305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:48.769305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:48.769305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:48.769305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:48.769305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:48.769305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:48.769305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:48.769305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:48.769305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:48.769305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:48.769305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:48.769305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:48.769305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:48.769305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:48.769305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:48.770290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:48.770290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:48.770290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:48.770290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:48.770290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:48.770290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:48.770290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:48.770290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:48.770290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:48.770290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:48.770290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:48.770290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:48.770290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:48.770290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:48.770290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:48.770290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:48.770290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:48.771192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:48.771192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:48.771192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:48.771192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 278/574] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:48.771192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7f7b2-4d84-4e88-bbc8-467819696140 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:48.810670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:48.872968 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea91796-3379-4ab3-ac26-3d00ab1fb315 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:48.874194 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7f7b2-4d84-4e88-bbc8-467819696140 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:48.890763 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:48.907378 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6ddcf-e675-423d-9e3b-d15c20e200c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:48.907614 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b96d9-3f38-489e-8e07-3023373a864a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:48.908006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6ddcf-e675-423d-9e3b-d15c20e200c2 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:48.908142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b96d9-3f38-489e-8e07-3023373a864a None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:48.908268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6ddcf-e675-423d-9e3b-d15c20e200c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:48.908369 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b96d9-3f38-489e-8e07-3023373a864a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:48.908562 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6ddcf-e675-423d-9e3b-d15c20e200c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:48.908721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b96d9-3f38-489e-8e07-3023373a864a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:48.909313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6ddcf-e675-423d-9e3b-d15c20e200c2 None admin] RBAC: Adding query filter params (role.id=2d62b8b8395745c7983c6c8d100d985b, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=fd88ec8b477443af95bff9af03471adf) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:48.909618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6ddcf-e675-423d-9e3b-d15c20e200c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:48.911269 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6ddcf-e675-423d-9e3b-d15c20e200c2 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:48.915185 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b96d9-3f38-489e-8e07-3023373a864a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:48.916357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 297/575] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:59:48 2022] GET /identity/v3/role_assignments?user.id=fd88ec8b477443af95bff9af03471adf&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2d62b8b8395745c7983c6c8d100d985b => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:48.916680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b96d9-3f38-489e-8e07-3023373a864a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:48.917138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6ddcf-e675-423d-9e3b-d15c20e200c2 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:48.929885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-eb3b96d9-3f38-489e-8e07-3023373a864a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:48.929885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:48.929885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:48.929885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:48.929885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:48.929885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:48.929885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:48.929885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:48.929885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:48.929885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:48.929885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:48.929885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:48.929885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:59:48.929885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:59:48.929885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:59:48.931036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:59:48.931036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:48.931036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:48.931036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:48.931036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:48.931036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:48.931036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:48.931036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:48.931036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:48.931036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:48.931036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:48.931036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:48.931036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:48.931036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:48.931036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:48.931036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:48.931036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:48.931971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:59:48.931971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:59:48.931971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:59:48.931971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:59:48.931971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:59:48.931971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:59:48.931971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:48.931971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:48.931971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 279/576] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:48.931971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b96d9-3f38-489e-8e07-3023373a864a None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:49.034025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6ddcf-e675-423d-9e3b-d15c20e200c2 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:49.042263 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:49.052469 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b96d9-3f38-489e-8e07-3023373a864a None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:49.074203 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37522f6e-52a0-4886-a87e-0b470d3788c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:49.074811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37522f6e-52a0-4886-a87e-0b470d3788c2 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:49.075085 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37522f6e-52a0-4886-a87e-0b470d3788c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:49.075358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37522f6e-52a0-4886-a87e-0b470d3788c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:49.076367 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37522f6e-52a0-4886-a87e-0b470d3788c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:49.076684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37522f6e-52a0-4886-a87e-0b470d3788c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:49.078024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37522f6e-52a0-4886-a87e-0b470d3788c2 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:49.086027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 298/577] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 06:59:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:49.086817 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37522f6e-52a0-4886-a87e-0b470d3788c2 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:49.095662 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d4652-5921-4190-ae5c-7d0340e831e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:49.096799 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d4652-5921-4190-ae5c-7d0340e831e5 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:49.097315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d4652-5921-4190-ae5c-7d0340e831e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:49.097771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d4652-5921-4190-ae5c-7d0340e831e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:49.098607 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d4652-5921-4190-ae5c-7d0340e831e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:49.099197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d4652-5921-4190-ae5c-7d0340e831e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:49.100946 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d4652-5921-4190-ae5c-7d0340e831e5 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:49.115888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 280/578] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:49.116973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d4652-5921-4190-ae5c-7d0340e831e5 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:49.165697 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:49.195366 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37522f6e-52a0-4886-a87e-0b470d3788c2 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:49.230662 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d4652-5921-4190-ae5c-7d0340e831e5 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:49.232910 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3873f-9dd2-4546-b61b-4387b31b82a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:49.233705 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3873f-9dd2-4546-b61b-4387b31b82a9 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:49.233986 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3873f-9dd2-4546-b61b-4387b31b82a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:49.234258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3873f-9dd2-4546-b61b-4387b31b82a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:49.245436 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3873f-9dd2-4546-b61b-4387b31b82a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:49.250617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-59f3873f-9dd2-4546-b61b-4387b31b82a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:49.250617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:49.250617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:49.250617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:49.250617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:49.250617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:49.250617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:49.250617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:49.250617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:49.250617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:49.250617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:49.250617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:49.250617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:49.250617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:49.250617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:49.250617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:49.251580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:49.251580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:49.251580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:49.251580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:49.251580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:49.251580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:49.251580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:49.251580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:49.251580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:49.251580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:49.251580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:49.251580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:49.251580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:49.251580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:49.251580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:49.251580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:49.252460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:49.252460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:49.252460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:49.252460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:49.252460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:49.252460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:49.252460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:49.252460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:49.252460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:49.252460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:49.252460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:49.252460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:49.252460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 299/579] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 06:59:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:49.253198 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3873f-9dd2-4546-b61b-4387b31b82a9 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:49.270382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b32bf1-5766-4cc4-90ba-0b102edb3d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:49.271020 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b32bf1-5766-4cc4-90ba-0b102edb3d5e None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:49.271295 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b32bf1-5766-4cc4-90ba-0b102edb3d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:49.271789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b32bf1-5766-4cc4-90ba-0b102edb3d5e None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:49.272287 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b32bf1-5766-4cc4-90ba-0b102edb3d5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:49.272585 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b32bf1-5766-4cc4-90ba-0b102edb3d5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:49.274012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b32bf1-5766-4cc4-90ba-0b102edb3d5e None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:49.280979 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:49.286515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 281/580] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 06:59:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:49.291997 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b32bf1-5766-4cc4-90ba-0b102edb3d5e None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:49.357493 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3873f-9dd2-4546-b61b-4387b31b82a9 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:49.391450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89c954-89d4-4877-bc64-b812ef251e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:49.392087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89c954-89d4-4877-bc64-b812ef251e27 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:49.392337 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89c954-89d4-4877-bc64-b812ef251e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:49.392604 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89c954-89d4-4877-bc64-b812ef251e27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:49.402710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89c954-89d4-4877-bc64-b812ef251e27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:49.405030 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89c954-89d4-4877-bc64-b812ef251e27 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:49.406527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b32bf1-5766-4cc4-90ba-0b102edb3d5e None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:49.408273 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:49.424776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-8e89c954-89d4-4877-bc64-b812ef251e27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:49.424776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:49.424776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:49.424776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:49.424776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:49.424776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:49.424776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:49.424776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:49.424776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:49.424776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:49.424776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:49.424776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:49.424776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:49.424776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:49.424776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:49.424776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:49.425894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:49.425894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:49.425894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:49.425894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:49.425894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:49.425894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:49.425894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:49.425894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:49.425894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:49.425894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:49.425894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:49.425894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:49.425894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:49.425894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:49.425894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:49.425894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:49.425894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:49.426752 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:49.426752 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:49.426752 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:49.427683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 300/581] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:49.428866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89c954-89d4-4877-bc64-b812ef251e27 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:49.446212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fa5d5-887d-45f4-8c92-c85466985f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:49.447076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fa5d5-887d-45f4-8c92-c85466985f6d None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:49.447566 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fa5d5-887d-45f4-8c92-c85466985f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:49.447978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fa5d5-887d-45f4-8c92-c85466985f6d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:49.456512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fa5d5-887d-45f4-8c92-c85466985f6d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:49.457964 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fa5d5-887d-45f4-8c92-c85466985f6d None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:49.470500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-c85fa5d5-887d-45f4-8c92-c85466985f6d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 06:59:49.470500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:49.470500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:49.470500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:49.470500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:49.470500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:49.470500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:49.470500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:49.470500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:49.470500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:49.470500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:49.470500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:49.470500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:59:49.470500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:59:49.470500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:59:49.470500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:59:49.471804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:49.471804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:49.471804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:59:49.471804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:49.471804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:59:49.471804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:49.471804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:49.471804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:49.471804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:49.471804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:49.471804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:49.471804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:49.471804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:49.471804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:49.471804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:49.471804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:49.471804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:49.471804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:49.472807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:49.472807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:49.472807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:59:49.472807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:59:49.472807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:59:49.472807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:59:49.472807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:59:49.472807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:59:49.472807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 06:59:49.472807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:49.472807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 282/582] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:49.472807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fa5d5-887d-45f4-8c92-c85466985f6d None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:49.518115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:49.541278 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89c954-89d4-4877-bc64-b812ef251e27 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:49.576471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fa5d5-887d-45f4-8c92-c85466985f6d None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:49.580882 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22f56d-3f93-4e0a-8475-7708c7448c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:49.581493 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22f56d-3f93-4e0a-8475-7708c7448c8c None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:49.581772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22f56d-3f93-4e0a-8475-7708c7448c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:49.582044 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22f56d-3f93-4e0a-8475-7708c7448c8c None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:49.582700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22f56d-3f93-4e0a-8475-7708c7448c8c None admin] RBAC: Adding query filter params (name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:49.583001 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22f56d-3f93-4e0a-8475-7708c7448c8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:49.584810 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22f56d-3f93-4e0a-8475-7708c7448c8c None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:49.592848 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 301/583] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 06:59:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:49.593739 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22f56d-3f93-4e0a-8475-7708c7448c8c None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:49.617652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57d425-7f25-4881-b5f3-2f3cd9d4e613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:49.618445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57d425-7f25-4881-b5f3-2f3cd9d4e613 None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:49.618828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57d425-7f25-4881-b5f3-2f3cd9d4e613 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:49.619222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57d425-7f25-4881-b5f3-2f3cd9d4e613 None admin] PATH_INFO: `/v3/projects/351a6777525c4207826d7df0d3768e99/users/a8d2cd93042c4583b63afe629d47fce0/roles/2d62b8b8395745c7983c6c8d100d985b` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:49.635173 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57d425-7f25-4881-b5f3-2f3cd9d4e613 None admin] RBAC: Authorizing `identity:create_grant(project_id=351a6777525c4207826d7df0d3768e99, user_id=a8d2cd93042c4583b63afe629d47fce0, role_id=2d62b8b8395745c7983c6c8d100d985b)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:49.636871 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57d425-7f25-4881-b5f3-2f3cd9d4e613 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:49.656775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 283/584] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 06:59:49 2022] PUT /identity/v3/projects/351a6777525c4207826d7df0d3768e99/users/a8d2cd93042c4583b63afe629d47fce0/roles/2d62b8b8395745c7983c6c8d100d985b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:59:49.657614 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57d425-7f25-4881-b5f3-2f3cd9d4e613 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:49.682294 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:49.714230 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22f56d-3f93-4e0a-8475-7708c7448c8c None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:49.760911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b2f93-806c-4f88-be2b-fd168476c764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:49.761969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b2f93-806c-4f88-be2b-fd168476c764 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:49.762952 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b2f93-806c-4f88-be2b-fd168476c764 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:49.763405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b2f93-806c-4f88-be2b-fd168476c764 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:49.764542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b2f93-806c-4f88-be2b-fd168476c764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:49.765053 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b2f93-806c-4f88-be2b-fd168476c764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:49.767744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b2f93-806c-4f88-be2b-fd168476c764 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:49.789015 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 302/585] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:49.790290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b2f93-806c-4f88-be2b-fd168476c764 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:49.795649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:49.808527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57d425-7f25-4881-b5f3-2f3cd9d4e613 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:49.851778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911644d-617b-40cc-80f7-f2b686082faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:49.852887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911644d-617b-40cc-80f7-f2b686082faa None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:49.853312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911644d-617b-40cc-80f7-f2b686082faa None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:49.853854 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911644d-617b-40cc-80f7-f2b686082faa None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:49.855110 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911644d-617b-40cc-80f7-f2b686082faa None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:49.855591 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911644d-617b-40cc-80f7-f2b686082faa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:49.857273 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911644d-617b-40cc-80f7-f2b686082faa None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:49.884115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 284/586] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Aug 23 06:59:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:49.885765 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911644d-617b-40cc-80f7-f2b686082faa None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:49.912536 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b2f93-806c-4f88-be2b-fd168476c764 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:49.949208 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbe789-18cd-4d9d-9cdd-682f67452ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:49.950049 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbe789-18cd-4d9d-9cdd-682f67452ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:49.950330 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbe789-18cd-4d9d-9cdd-682f67452ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:49.950906 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbe789-18cd-4d9d-9cdd-682f67452ed4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:49.964110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbe789-18cd-4d9d-9cdd-682f67452ed4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:49.965598 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbe789-18cd-4d9d-9cdd-682f67452ed4 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:49.970717 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:49.985721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-60bbe789-18cd-4d9d-9cdd-682f67452ed4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:49.985721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:49.985721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:49.985721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:49.985721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:49.985721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:49.985721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:49.985721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:49.985721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:49.985721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:49.985721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:49.985721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:49.985721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:49.985721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:49.985721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:49.985721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:49.987050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:49.987050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:49.987050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:49.987050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:49.987050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:49.987050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:49.987050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:49.987050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:49.987050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:49.987050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:49.987050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:49.987050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:49.987050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:49.987050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:49.987050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:49.987050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:49.987050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:49.988550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:49.988550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:49.988550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:49.988550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 303/587] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:49.992948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbe789-18cd-4d9d-9cdd-682f67452ed4 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:50.031259 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911644d-617b-40cc-80f7-f2b686082faa None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:50.068821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa6669-525f-4436-a11a-55085dea9211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:50.069737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6669-525f-4436-a11a-55085dea9211 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:50.070019 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6669-525f-4436-a11a-55085dea9211 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:50.070628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6669-525f-4436-a11a-55085dea9211 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:50.074320 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:50.078562 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa6669-525f-4436-a11a-55085dea9211 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:50.080063 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa6669-525f-4436-a11a-55085dea9211 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:50.093546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-b9fa6669-525f-4436-a11a-55085dea9211 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 06:59:50.093546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:50.093546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:50.093546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:50.093546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:50.093546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:50.093546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:50.093546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:50.093546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:50.093546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:50.093546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:50.093546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:50.093546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:59:50.093546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:59:50.093546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:59:50.093546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:59:50.094683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:50.094683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:50.094683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:59:50.094683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:50.094683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:59:50.094683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:50.094683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:50.094683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:50.094683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:50.094683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:50.094683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:50.094683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:50.094683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:50.094683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:50.094683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:50.094683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:50.094683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:50.094683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:50.095685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:50.095685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:50.095685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:59:50.095685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:59:50.095685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:59:50.095685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:59:50.095685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:59:50.095685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:59:50.095685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 06:59:50.095685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:50.095685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 285/588] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:50.095685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa6669-525f-4436-a11a-55085dea9211 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:50.136869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbe789-18cd-4d9d-9cdd-682f67452ed4 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:50.183201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4764b6-1eab-432f-891c-a712e88dc4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:50.184109 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4764b6-1eab-432f-891c-a712e88dc4fe None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:50.184589 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4764b6-1eab-432f-891c-a712e88dc4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:50.185419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4764b6-1eab-432f-891c-a712e88dc4fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:50.201202 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4764b6-1eab-432f-891c-a712e88dc4fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:50.203869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4764b6-1eab-432f-891c-a712e88dc4fe None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:50.214577 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa6669-525f-4436-a11a-55085dea9211 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:50.220542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-ac4764b6-1eab-432f-891c-a712e88dc4fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:50.220542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:50.220542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:50.220542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:50.220542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:50.220542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:50.220542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:50.220542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:50.220542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:50.220542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:50.220542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:50.220542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:50.220542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:50.220542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:50.220542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:50.220542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:50.221760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:50.221760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:50.221760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:50.221760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:50.221760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:50.221760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:50.221760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:50.221760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:50.221760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:50.221760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:50.221760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:50.221760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:50.221760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:50.221760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:50.221760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:50.221760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:50.221760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:50.222956 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:50.222956 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:50.222956 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:50.227742 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 304/589] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:50.228724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4764b6-1eab-432f-891c-a712e88dc4fe None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:50.250654 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:50.263147 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f9efa-7214-4c99-be35-db24e6184400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:50.264256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f9efa-7214-4c99-be35-db24e6184400 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:50.264708 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f9efa-7214-4c99-be35-db24e6184400 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:50.265521 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f9efa-7214-4c99-be35-db24e6184400 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:50.266588 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f9efa-7214-4c99-be35-db24e6184400 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:50.267039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f9efa-7214-4c99-be35-db24e6184400 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:50.269012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f9efa-7214-4c99-be35-db24e6184400 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:50.285691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 286/590] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:50.286736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f9efa-7214-4c99-be35-db24e6184400 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:50.345774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4764b6-1eab-432f-891c-a712e88dc4fe None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:50.354669 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:50.386249 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e28a1-f236-4f91-9739-3be5209e9b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:50.386692 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e28a1-f236-4f91-9739-3be5209e9b9b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:50.387069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e28a1-f236-4f91-9739-3be5209e9b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:50.387504 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e28a1-f236-4f91-9739-3be5209e9b9b None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:50.388391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e28a1-f236-4f91-9739-3be5209e9b9b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:50.388795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e28a1-f236-4f91-9739-3be5209e9b9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:50.390316 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e28a1-f236-4f91-9739-3be5209e9b9b None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:50.413904 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 305/591] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Aug 23 06:59:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:50.414949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e28a1-f236-4f91-9739-3be5209e9b9b None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:50.422917 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f9efa-7214-4c99-be35-db24e6184400 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:50.463382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de0c17-8e77-4de3-8ed9-347a79b55d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:50.464117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de0c17-8e77-4de3-8ed9-347a79b55d57 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:50.464492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de0c17-8e77-4de3-8ed9-347a79b55d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:50.464862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de0c17-8e77-4de3-8ed9-347a79b55d57 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:50.465658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de0c17-8e77-4de3-8ed9-347a79b55d57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:50.466056 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de0c17-8e77-4de3-8ed9-347a79b55d57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:50.467605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de0c17-8e77-4de3-8ed9-347a79b55d57 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:50.485173 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 287/592] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:50.486200 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de0c17-8e77-4de3-8ed9-347a79b55d57 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:50.537587 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:50.547370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e28a1-f236-4f91-9739-3be5209e9b9b None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:50.595088 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efa55f-c4a0-4351-9fca-6a2e0a37b248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:50.595629 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de0c17-8e77-4de3-8ed9-347a79b55d57 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:50.596186 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efa55f-c4a0-4351-9fca-6a2e0a37b248 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:50.596732 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efa55f-c4a0-4351-9fca-6a2e0a37b248 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:50.597252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efa55f-c4a0-4351-9fca-6a2e0a37b248 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:50.607028 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32efa55f-c4a0-4351-9fca-6a2e0a37b248 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:50.609420 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32efa55f-c4a0-4351-9fca-6a2e0a37b248 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:50.622138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-32efa55f-c4a0-4351-9fca-6a2e0a37b248 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 06:59:50.622138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:50.622138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:50.622138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:50.622138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:50.622138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:50.622138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:50.622138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:50.622138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:50.622138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:50.622138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:50.622138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:50.622138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:59:50.622138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:59:50.622138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:59:50.622138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:59:50.623301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:50.623301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:50.623301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:59:50.623301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:50.623301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:59:50.623301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:50.623301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:50.623301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:50.623301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:50.623301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:50.623301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:50.623301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:50.623301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:50.623301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:50.623301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:50.623301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:50.623301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:50.623301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:50.624498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:50.624498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:50.624498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:59:50.624498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:59:50.624498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:59:50.624498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:59:50.624498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:59:50.624498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:59:50.624498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 06:59:50.624498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:50.630829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 306/593] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 06:59:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:50.631814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efa55f-c4a0-4351-9fca-6a2e0a37b248 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:50.638208 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e6c40-3e02-484c-b3f5-11f996c1ff32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:50.639034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e6c40-3e02-484c-b3f5-11f996c1ff32 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:50.639493 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e6c40-3e02-484c-b3f5-11f996c1ff32 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:50.639911 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e6c40-3e02-484c-b3f5-11f996c1ff32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:50.658560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e6c40-3e02-484c-b3f5-11f996c1ff32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:50.660332 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e6c40-3e02-484c-b3f5-11f996c1ff32 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:50.668072 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:50.684097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-e88e6c40-3e02-484c-b3f5-11f996c1ff32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:50.684097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:50.684097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:50.684097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:50.684097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:50.684097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:50.684097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:50.684097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:50.684097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:50.684097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:50.684097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:50.684097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:50.684097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:50.684097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:50.684097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:50.684097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:50.685114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:50.685114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:50.685114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:50.685114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:50.685114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:50.685114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:50.685114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:50.685114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:50.685114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:50.685114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:50.685114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:50.685114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:50.685114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:50.685114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:50.685114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:50.685114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:50.685114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:50.685988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:50.685988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:50.685988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:50.685988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 288/594] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:50.686222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e6c40-3e02-484c-b3f5-11f996c1ff32 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:50.779613 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efa55f-c4a0-4351-9fca-6a2e0a37b248 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:50.797449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:50.805228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e6c40-3e02-484c-b3f5-11f996c1ff32 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:50.819021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb881ee2-310d-47c4-bb1c-d5845b5f5f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:50.819690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb881ee2-310d-47c4-bb1c-d5845b5f5f7b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:50.819964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb881ee2-310d-47c4-bb1c-d5845b5f5f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:50.820227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb881ee2-310d-47c4-bb1c-d5845b5f5f7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:50.827475 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb881ee2-310d-47c4-bb1c-d5845b5f5f7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:50.828925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb881ee2-310d-47c4-bb1c-d5845b5f5f7b None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:50.840594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-eb881ee2-310d-47c4-bb1c-d5845b5f5f7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:50.840594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:50.840594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:50.840594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:50.840594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:50.840594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:50.840594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:50.840594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:50.840594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:50.840594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:50.840594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:50.840594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:50.840594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:59:50.840594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:59:50.840594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:59:50.841654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:59:50.841654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:50.841654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:50.841654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:50.841654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:50.841654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:50.841654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:50.841654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:50.841654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:50.841654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:50.841654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:50.841654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:50.841654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:50.841654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:50.841654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:50.841654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:50.841654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:50.842710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:59:50.842710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:59:50.842710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:59:50.842710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:59:50.842710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:59:50.842710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:59:50.842710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:50.842710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:50.842710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 307/595] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:50.842710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb881ee2-310d-47c4-bb1c-d5845b5f5f7b None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:50.851294 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25c468-5349-49a9-8314-c017ed748a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:50.852186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25c468-5349-49a9-8314-c017ed748a6c None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:50.852329 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25c468-5349-49a9-8314-c017ed748a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:50.852487 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25c468-5349-49a9-8314-c017ed748a6c None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:50.853225 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25c468-5349-49a9-8314-c017ed748a6c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:50.853531 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25c468-5349-49a9-8314-c017ed748a6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:50.854984 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25c468-5349-49a9-8314-c017ed748a6c None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:50.872814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 289/596] 10.222.0.17 () {56 vars in 1139 bytes} [Tue Aug 23 06:59:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:50.872814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862d350-5d5b-4362-b6bf-e2186041bb2e None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:50.872814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862d350-5d5b-4362-b6bf-e2186041bb2e None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:50.872814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862d350-5d5b-4362-b6bf-e2186041bb2e None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:50.872814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 290/597] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:50.872814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862d350-5d5b-4362-b6bf-e2186041bb2e None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:50.935290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:50.951535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb881ee2-310d-47c4-bb1c-d5845b5f5f7b None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:50.985991 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e210bc-b25d-4214-a977-963f1e992b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:50.986601 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e210bc-b25d-4214-a977-963f1e992b3e None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:50.986880 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e210bc-b25d-4214-a977-963f1e992b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:50.987209 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862d350-5d5b-4362-b6bf-e2186041bb2e None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:50.987317 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e210bc-b25d-4214-a977-963f1e992b3e None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:50.987935 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e210bc-b25d-4214-a977-963f1e992b3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:50.988238 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e210bc-b25d-4214-a977-963f1e992b3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:50.989633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e210bc-b25d-4214-a977-963f1e992b3e None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:51.006598 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 308/598] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:51.006598 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e210bc-b25d-4214-a977-963f1e992b3e None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:51.030787 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b55eba-bab6-428a-be1f-47a9a24b7da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:51.031448 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b55eba-bab6-428a-be1f-47a9a24b7da8 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:51.031727 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b55eba-bab6-428a-be1f-47a9a24b7da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:51.032105 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b55eba-bab6-428a-be1f-47a9a24b7da8 None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:51.032800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b55eba-bab6-428a-be1f-47a9a24b7da8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:51.033102 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b55eba-bab6-428a-be1f-47a9a24b7da8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:51.034756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b55eba-bab6-428a-be1f-47a9a24b7da8 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:51.051438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 291/599] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 06:59:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:51.051438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b5b8d-c934-422b-becf-1cf153e08bc4 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:51.051438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b5b8d-c934-422b-becf-1cf153e08bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:51.051438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b5b8d-c934-422b-becf-1cf153e08bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:51.067230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-852b5b8d-c934-422b-becf-1cf153e08bc4 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:51.068617 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-852b5b8d-c934-422b-becf-1cf153e08bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:51.080577 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:51.107827 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e210bc-b25d-4214-a977-963f1e992b3e None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:51.151439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 292/600] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:51.151439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b5b8d-c934-422b-becf-1cf153e08bc4 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:51.152576 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e8f2b-1b8f-4e14-b4c7-2eeccfd69e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:51.153377 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e8f2b-1b8f-4e14-b4c7-2eeccfd69e71 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:51.153786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e8f2b-1b8f-4e14-b4c7-2eeccfd69e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:51.154125 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e8f2b-1b8f-4e14-b4c7-2eeccfd69e71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:51.166106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e8f2b-1b8f-4e14-b4c7-2eeccfd69e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:51.167679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e8f2b-1b8f-4e14-b4c7-2eeccfd69e71 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:51.170514 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:51.185252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-511e8f2b-1b8f-4e14-b4c7-2eeccfd69e71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:51.185252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:51.185252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:51.185252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:51.185252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:51.185252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:51.185252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:51.185252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:51.185252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:51.185252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:51.185252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:51.185252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:51.185252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:51.185252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:51.185252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:51.185252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:51.186204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:51.186204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:51.186204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:51.186204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:51.186204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:51.186204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:51.186204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:51.186204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:51.186204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:51.186204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:51.186204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:51.186204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:51.186204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:51.186204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:51.186204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:51.186204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:51.186204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:51.187086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:51.187086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:51.187086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:51.191904 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 309/601] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:51.192804 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e8f2b-1b8f-4e14-b4c7-2eeccfd69e71 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:51.267004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b5b8d-c934-422b-becf-1cf153e08bc4 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:51.306117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e3e34-5317-444e-88a7-ef99b4c0f4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:51.306678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e3e34-5317-444e-88a7-ef99b4c0f4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:51.306955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e3e34-5317-444e-88a7-ef99b4c0f4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:51.307227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e3e34-5317-444e-88a7-ef99b4c0f4e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:51.311139 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e8f2b-1b8f-4e14-b4c7-2eeccfd69e71 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:51.314889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e3e34-5317-444e-88a7-ef99b4c0f4e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:51.316165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e3e34-5317-444e-88a7-ef99b4c0f4e5 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:51.328497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-a37e3e34-5317-444e-88a7-ef99b4c0f4e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:51.328497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:51.328497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:51.328497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:51.328497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:51.328497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:51.328497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:51.328497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:51.328497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:51.328497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:51.328497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:51.328497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:51.328497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:59:51.328497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:59:51.328497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:59:51.329404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:59:51.329404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:51.329404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:51.329404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:51.329404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:51.329404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:51.329404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:51.329404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:51.329404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:51.329404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:51.329404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:51.329404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:51.329404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:51.329404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:51.329404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:51.329404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:51.329404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:51.330887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:59:51.330887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:59:51.330887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:59:51.330887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:59:51.330887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:59:51.330887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:59:51.330887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:51.330887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:51.330887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 293/602] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:51.331369 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e3e34-5317-444e-88a7-ef99b4c0f4e5 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:51.340557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:51.355709 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab26b4-376a-4fdd-8ed0-fe20aa5ef2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:51.356682 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab26b4-376a-4fdd-8ed0-fe20aa5ef2f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:51.357154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab26b4-376a-4fdd-8ed0-fe20aa5ef2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:51.358006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab26b4-376a-4fdd-8ed0-fe20aa5ef2f8 None admin] PATH_INFO: `/v3/projects/351a6777525c4207826d7df0d3768e99/users/57a5f04cb9674b86a6b4b9a472e1afc0/roles/2d62b8b8395745c7983c6c8d100d985b` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:51.376455 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab26b4-376a-4fdd-8ed0-fe20aa5ef2f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=351a6777525c4207826d7df0d3768e99, user_id=57a5f04cb9674b86a6b4b9a472e1afc0, role_id=2d62b8b8395745c7983c6c8d100d985b)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:51.378622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab26b4-376a-4fdd-8ed0-fe20aa5ef2f8 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:51.396852 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 310/603] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 06:59:51 2022] PUT /identity/v3/projects/351a6777525c4207826d7df0d3768e99/users/57a5f04cb9674b86a6b4b9a472e1afc0/roles/2d62b8b8395745c7983c6c8d100d985b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:59:51.398108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab26b4-376a-4fdd-8ed0-fe20aa5ef2f8 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:51.460817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e3e34-5317-444e-88a7-ef99b4c0f4e5 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:51.462264 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:51.509408 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9d563-9dcd-4954-8590-b38b3ff7ff2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:51.510276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9d563-9dcd-4954-8590-b38b3ff7ff2f None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:51.510642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9d563-9dcd-4954-8590-b38b3ff7ff2f None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:51.511039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9d563-9dcd-4954-8590-b38b3ff7ff2f None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:51.511860 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9d563-9dcd-4954-8590-b38b3ff7ff2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:51.512270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9d563-9dcd-4954-8590-b38b3ff7ff2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:51.513851 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9d563-9dcd-4954-8590-b38b3ff7ff2f None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:51.528232 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 294/604] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:51.529519 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9d563-9dcd-4954-8590-b38b3ff7ff2f None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:51.539781 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab26b4-376a-4fdd-8ed0-fe20aa5ef2f8 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:51.580255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bb44d-3efd-4375-8aef-5b8234968570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:51.581140 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bb44d-3efd-4375-8aef-5b8234968570 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:51.581555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bb44d-3efd-4375-8aef-5b8234968570 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:51.581982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bb44d-3efd-4375-8aef-5b8234968570 None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:51.582833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bb44d-3efd-4375-8aef-5b8234968570 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:51.583273 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bb44d-3efd-4375-8aef-5b8234968570 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:51.584920 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bb44d-3efd-4375-8aef-5b8234968570 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:51.598386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 311/605] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 06:59:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:51.599313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bb44d-3efd-4375-8aef-5b8234968570 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:51.621019 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:51.666133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9d563-9dcd-4954-8590-b38b3ff7ff2f None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:51.704444 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96605256-b22c-4cdf-bf84-da1d3022af7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:51.705252 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96605256-b22c-4cdf-bf84-da1d3022af7e None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:51.705527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96605256-b22c-4cdf-bf84-da1d3022af7e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:51.705805 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96605256-b22c-4cdf-bf84-da1d3022af7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:51.709012 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bb44d-3efd-4375-8aef-5b8234968570 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:51.716656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96605256-b22c-4cdf-bf84-da1d3022af7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:51.716921 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:51.721306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-96605256-b22c-4cdf-bf84-da1d3022af7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:51.721306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:51.721306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:51.721306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:51.721306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:51.721306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:51.721306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:51.721306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:51.721306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:51.721306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:51.721306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:51.721306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:51.721306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:51.721306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:51.721306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:51.721306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:51.722336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:51.722336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:51.722336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:51.722336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:51.722336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:51.722336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:51.722336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:51.722336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:51.722336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:51.722336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:51.722336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:51.722336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:51.722336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:51.722336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:51.722336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:51.722336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:51.723189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:51.723189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:51.723189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:51.723189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:51.723189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:51.723189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:51.723189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:51.723189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:51.723189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:51.723189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:51.723189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:51.723189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:51.723189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 295/606] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 06:59:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:51.723944 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96605256-b22c-4cdf-bf84-da1d3022af7e None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:51.752022 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbf2e2-c2c2-4959-b3fb-eb2c89437855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:51.752653 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbf2e2-c2c2-4959-b3fb-eb2c89437855 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:51.752959 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbf2e2-c2c2-4959-b3fb-eb2c89437855 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:51.753227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbf2e2-c2c2-4959-b3fb-eb2c89437855 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:51.760433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbf2e2-c2c2-4959-b3fb-eb2c89437855 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:51.762305 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbf2e2-c2c2-4959-b3fb-eb2c89437855 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:51.774690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-6bcbf2e2-c2c2-4959-b3fb-eb2c89437855 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:51.774690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:51.774690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:51.774690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:51.774690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:51.774690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:51.774690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:51.774690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:51.774690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:51.774690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:51.774690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:51.774690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:51.774690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:59:51.774690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:59:51.774690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:59:51.775824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:59:51.775824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:51.775824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:51.775824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:51.775824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:51.775824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:51.775824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:51.775824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:51.775824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:51.775824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:51.775824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:51.775824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:51.775824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:51.775824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:51.775824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:51.775824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:51.775824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:51.776960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:59:51.776960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:59:51.776960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:59:51.776960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:59:51.776960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:59:51.776960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:59:51.776960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:51.776960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:51.779357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 312/607] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:51.781648 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1797c16-ff6c-4274-862f-558f96d87b2c None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:51.782077 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1797c16-ff6c-4274-862f-558f96d87b2c None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:51.782501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1797c16-ff6c-4274-862f-558f96d87b2c None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:51.784667 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 313/608] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:51.788171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1797c16-ff6c-4274-862f-558f96d87b2c None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:51.836791 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96605256-b22c-4cdf-bf84-da1d3022af7e None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:51.862440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:51.871532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9632807-483d-4d81-9e99-69476600cd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:51.872274 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9632807-483d-4d81-9e99-69476600cd9d None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:51.872619 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9632807-483d-4d81-9e99-69476600cd9d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:51.872957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9632807-483d-4d81-9e99-69476600cd9d None admin] PATH_INFO: `/v3/projects/351a6777525c4207826d7df0d3768e99/users/a26f09e892a74d9eab97e73cce9d062a/roles/2d62b8b8395745c7983c6c8d100d985b` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:51.888168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9632807-483d-4d81-9e99-69476600cd9d None admin] RBAC: Authorizing `identity:create_grant(project_id=351a6777525c4207826d7df0d3768e99, user_id=a26f09e892a74d9eab97e73cce9d062a, role_id=2d62b8b8395745c7983c6c8d100d985b)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:51.889675 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9632807-483d-4d81-9e99-69476600cd9d None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:51.903323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1797c16-ff6c-4274-862f-558f96d87b2c None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:51.904477 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 296/609] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 06:59:51 2022] PUT /identity/v3/projects/351a6777525c4207826d7df0d3768e99/users/a26f09e892a74d9eab97e73cce9d062a/roles/2d62b8b8395745c7983c6c8d100d985b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:59:51.905314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9632807-483d-4d81-9e99-69476600cd9d None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:51.964390 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b19077-2093-4b81-8f93-a56e9f50d1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:51.965019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b19077-2093-4b81-8f93-a56e9f50d1bf None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:51.971441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b19077-2093-4b81-8f93-a56e9f50d1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:51.971441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b19077-2093-4b81-8f93-a56e9f50d1bf None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:51.971441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b19077-2093-4b81-8f93-a56e9f50d1bf None admin] RBAC: Adding query filter params (name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:51.971441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b19077-2093-4b81-8f93-a56e9f50d1bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:51.971441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b19077-2093-4b81-8f93-a56e9f50d1bf None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:51.983392 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 314/610] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 06:59:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:51.984872 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97025d-2c53-492d-8be1-69a392d7ab4e None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:51.985161 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97025d-2c53-492d-8be1-69a392d7ab4e None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:51.985430 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97025d-2c53-492d-8be1-69a392d7ab4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:52.009011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-af97025d-2c53-492d-8be1-69a392d7ab4e None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:52.010510 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-af97025d-2c53-492d-8be1-69a392d7ab4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:52.038592 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9632807-483d-4d81-9e99-69476600cd9d None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:52.042223 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:52.080005 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19d17a-0fb4-4490-b933-8cb41e57a44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:52.080624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19d17a-0fb4-4490-b933-8cb41e57a44f None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:52.081198 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19d17a-0fb4-4490-b933-8cb41e57a44f None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:52.081198 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19d17a-0fb4-4490-b933-8cb41e57a44f None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:52.081905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e19d17a-0fb4-4490-b933-8cb41e57a44f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:52.082216 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e19d17a-0fb4-4490-b933-8cb41e57a44f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:52.083703 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e19d17a-0fb4-4490-b933-8cb41e57a44f None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:52.093152 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 315/611] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:52.093946 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97025d-2c53-492d-8be1-69a392d7ab4e None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:52.096046 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 297/612] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 06:59:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:52.103270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19d17a-0fb4-4490-b933-8cb41e57a44f None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:52.152186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:52.208163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97025d-2c53-492d-8be1-69a392d7ab4e None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:52.236956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19d17a-0fb4-4490-b933-8cb41e57a44f None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:52.244787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16d6d3-b91f-45e4-a58d-7e4dfb518c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:52.245545 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16d6d3-b91f-45e4-a58d-7e4dfb518c3b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:52.245916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16d6d3-b91f-45e4-a58d-7e4dfb518c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:52.246349 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16d6d3-b91f-45e4-a58d-7e4dfb518c3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:52.269173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b16d6d3-b91f-45e4-a58d-7e4dfb518c3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:52.270632 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b16d6d3-b91f-45e4-a58d-7e4dfb518c3b None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:52.292358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-7b16d6d3-b91f-45e4-a58d-7e4dfb518c3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:52.292358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:52.292358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:52.292358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:52.292358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:52.292358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:52.292358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:52.292358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:52.292358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:52.292358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:52.292358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:52.292358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:52.292358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:52.292358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:52.292358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:52.292358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:52.293311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:52.293311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:52.293311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:52.293311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:52.293311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:52.293311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:52.293311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:52.293311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:52.293311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:52.293311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:52.293311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:52.293311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:52.293311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:52.293311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:52.293311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:52.293311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:52.293311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:52.294191 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:52.294191 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:52.294191 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:52.295511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 316/613] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:52.298611 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8509dda-c097-4a29-8464-681e6075729c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:52.299273 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8509dda-c097-4a29-8464-681e6075729c None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:52.299584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8509dda-c097-4a29-8464-681e6075729c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:52.299910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8509dda-c097-4a29-8464-681e6075729c None admin] PATH_INFO: `/v3/projects/351a6777525c4207826d7df0d3768e99/users/938364c02d8d4dd2ae832c5129be3cbe/roles/2d62b8b8395745c7983c6c8d100d985b` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:52.302640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16d6d3-b91f-45e4-a58d-7e4dfb518c3b None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:52.330785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:52.336306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8509dda-c097-4a29-8464-681e6075729c None admin] RBAC: Authorizing `identity:create_grant(project_id=351a6777525c4207826d7df0d3768e99, user_id=938364c02d8d4dd2ae832c5129be3cbe, role_id=2d62b8b8395745c7983c6c8d100d985b)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:52.339443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8509dda-c097-4a29-8464-681e6075729c None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:52.358645 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 298/614] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 06:59:52 2022] PUT /identity/v3/projects/351a6777525c4207826d7df0d3768e99/users/938364c02d8d4dd2ae832c5129be3cbe/roles/2d62b8b8395745c7983c6c8d100d985b => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 06:59:52.425979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8509dda-c097-4a29-8464-681e6075729c None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:52.466418 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16d6d3-b91f-45e4-a58d-7e4dfb518c3b None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:52.539157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e0dc9-04c8-425a-b048-9486b6817049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:52.540130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e0dc9-04c8-425a-b048-9486b6817049 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:52.540592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e0dc9-04c8-425a-b048-9486b6817049 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:52.541039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e0dc9-04c8-425a-b048-9486b6817049 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:52.541885 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e0dc9-04c8-425a-b048-9486b6817049 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:52.542414 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e0dc9-04c8-425a-b048-9486b6817049 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:52.544116 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e0dc9-04c8-425a-b048-9486b6817049 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:52.566988 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 317/615] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:52.572291 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e0dc9-04c8-425a-b048-9486b6817049 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:52.575233 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:52.594590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8509dda-c097-4a29-8464-681e6075729c None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:52.659851 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb045c2-3218-4055-8d79-3157e32b7f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:52.660566 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb045c2-3218-4055-8d79-3157e32b7f5b None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:52.660842 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb045c2-3218-4055-8d79-3157e32b7f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:52.661112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb045c2-3218-4055-8d79-3157e32b7f5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:52.666480 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:52.679585 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb045c2-3218-4055-8d79-3157e32b7f5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:52.680061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-fdb045c2-3218-4055-8d79-3157e32b7f5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:52.680061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:52.680061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:52.680061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:52.680061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:52.680061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:52.680061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:52.680061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:52.680061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:52.680061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:52.680061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:52.680061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:52.680061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:52.680061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:52.680061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:52.680061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:52.680979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:52.680979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:52.680979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:52.680979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:52.680979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:52.680979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:52.680979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:52.680979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:52.680979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:52.680979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:52.680979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:52.680979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:52.680979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:52.680979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:52.680979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:52.680979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:52.681841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:52.681841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:52.681841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:52.681841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:52.681841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:52.681841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:52.681841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:52.681841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:52.681841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:52.681841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:52.681841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:52.681841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:52.681841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 299/616] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 06:59:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:52.687302 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb045c2-3218-4055-8d79-3157e32b7f5b None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:52.729180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e0dc9-04c8-425a-b048-9486b6817049 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:52.790616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f736aa1-8bdd-470d-9a3d-165dd92024c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:52.791253 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f736aa1-8bdd-470d-9a3d-165dd92024c0 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:52.791557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f736aa1-8bdd-470d-9a3d-165dd92024c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:52.791829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f736aa1-8bdd-470d-9a3d-165dd92024c0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:52.805548 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f736aa1-8bdd-470d-9a3d-165dd92024c0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:52.807006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f736aa1-8bdd-470d-9a3d-165dd92024c0 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:52.833236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-7f736aa1-8bdd-470d-9a3d-165dd92024c0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 06:59:52.833236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:52.833236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:52.833236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:52.833236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:52.833236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:52.833236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:52.833236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:52.833236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:52.833236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:52.833236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:52.833236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:52.833236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:59:52.833236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:59:52.833236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:59:52.833236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:59:52.834221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:52.834221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:52.834221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:59:52.834221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:52.834221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:59:52.834221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:52.834221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:52.834221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:52.834221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:52.834221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:52.834221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:52.834221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:52.834221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:52.834221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:52.834221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:52.834221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:52.834221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:52.834221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:52.835179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:52.835179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:52.835179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:59:52.835179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:59:52.835179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:59:52.835179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:59:52.835179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:59:52.835179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:59:52.835179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 06:59:52.835179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:52.835179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 318/617] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 06:59:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:52.839446 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb045c2-3218-4055-8d79-3157e32b7f5b None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:52.844384 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f736aa1-8bdd-470d-9a3d-165dd92024c0 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:52.875921 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e177c32-d87d-4cc7-a38d-1d2c61092565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:52.876540 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e177c32-d87d-4cc7-a38d-1d2c61092565 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:52.877562 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e177c32-d87d-4cc7-a38d-1d2c61092565 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:52.877988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e177c32-d87d-4cc7-a38d-1d2c61092565 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:52.878698 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e177c32-d87d-4cc7-a38d-1d2c61092565 None admin] RBAC: Adding query filter params (name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:52.878985 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e177c32-d87d-4cc7-a38d-1d2c61092565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:52.880683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e177c32-d87d-4cc7-a38d-1d2c61092565 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:52.900184 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 300/618] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 06:59:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:52.905162 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e177c32-d87d-4cc7-a38d-1d2c61092565 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:52.925732 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:53.013895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f736aa1-8bdd-470d-9a3d-165dd92024c0 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:53.060625 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:53.070221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82312d-f3c2-4c4f-86d6-138b7f4593a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:53.071587 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82312d-f3c2-4c4f-86d6-138b7f4593a2 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:53.071864 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82312d-f3c2-4c4f-86d6-138b7f4593a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:53.072133 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82312d-f3c2-4c4f-86d6-138b7f4593a2 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:53.073096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82312d-f3c2-4c4f-86d6-138b7f4593a2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:53.073398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82312d-f3c2-4c4f-86d6-138b7f4593a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:53.074827 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82312d-f3c2-4c4f-86d6-138b7f4593a2 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:53.104453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e177c32-d87d-4cc7-a38d-1d2c61092565 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:53.115765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 319/619] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Aug 23 06:59:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:53.122194 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82312d-f3c2-4c4f-86d6-138b7f4593a2 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:53.146267 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d2308-859e-45e6-92a8-d39f81ab9feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:53.146913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d2308-859e-45e6-92a8-d39f81ab9feb None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:53.147188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d2308-859e-45e6-92a8-d39f81ab9feb None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:53.147501 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d2308-859e-45e6-92a8-d39f81ab9feb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:53.158847 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d2308-859e-45e6-92a8-d39f81ab9feb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:53.160498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d2308-859e-45e6-92a8-d39f81ab9feb None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:53.176132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-d25d2308-859e-45e6-92a8-d39f81ab9feb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:53.176132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:53.176132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:53.176132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:53.176132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:53.176132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:53.176132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:53.176132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:53.176132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:53.176132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:53.176132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:53.176132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:53.176132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:53.176132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:53.176132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:53.176132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:53.177249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:53.177249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:53.177249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:53.177249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:53.177249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:53.177249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:53.177249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:53.177249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:53.177249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:53.177249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:53.177249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:53.177249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:53.177249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:53.177249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:53.177249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:53.177249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:53.177249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:53.178157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:53.178157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:53.178157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:53.178157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 301/620] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:53.183095 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d2308-859e-45e6-92a8-d39f81ab9feb None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:53.305213 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:53.349582 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82312d-f3c2-4c4f-86d6-138b7f4593a2 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:53.381045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d2308-859e-45e6-92a8-d39f81ab9feb None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:53.418097 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7162996-48d9-487c-9524-b16692bbcccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:53.418732 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7162996-48d9-487c-9524-b16692bbcccc None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:53.419015 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7162996-48d9-487c-9524-b16692bbcccc None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:53.419285 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7162996-48d9-487c-9524-b16692bbcccc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:53.432728 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7162996-48d9-487c-9524-b16692bbcccc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:53.434118 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7162996-48d9-487c-9524-b16692bbcccc None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:53.450246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-b7162996-48d9-487c-9524-b16692bbcccc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:53.450246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:53.450246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:53.450246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:53.450246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:53.450246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:53.450246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:53.450246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:53.450246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:53.450246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:53.450246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:53.450246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:53.450246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:53.450246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:53.450246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:53.450246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:53.451556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:53.451556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:53.451556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:53.451556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:53.451556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:53.451556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:53.451556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:53.451556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:53.451556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:53.451556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:53.451556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:53.451556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:53.451556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:53.451556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:53.451556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:53.451556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:53.451556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:53.452452 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:53.452452 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:53.452452 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:53.452452 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 320/621] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:53.458027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7162996-48d9-487c-9524-b16692bbcccc None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:53.460221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801dbfad-96d7-4553-953c-54ce5cc82fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:53.461696 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801dbfad-96d7-4553-953c-54ce5cc82fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:53.462229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801dbfad-96d7-4553-953c-54ce5cc82fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:53.462509 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801dbfad-96d7-4553-953c-54ce5cc82fe9 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:53.463172 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801dbfad-96d7-4553-953c-54ce5cc82fe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:53.464217 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801dbfad-96d7-4553-953c-54ce5cc82fe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:53.465616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801dbfad-96d7-4553-953c-54ce5cc82fe9 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:53.491110 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 302/622] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:53.496492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801dbfad-96d7-4553-953c-54ce5cc82fe9 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:53.542123 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:53.611609 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7162996-48d9-487c-9524-b16692bbcccc None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:53.622509 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801dbfad-96d7-4553-953c-54ce5cc82fe9 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:53.644877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:53.649013 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992cbd68-94eb-460d-bb25-79dc70dfd3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:53.649796 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cbd68-94eb-460d-bb25-79dc70dfd3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:53.650217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cbd68-94eb-460d-bb25-79dc70dfd3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:53.650627 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cbd68-94eb-460d-bb25-79dc70dfd3c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:53.651464 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992cbd68-94eb-460d-bb25-79dc70dfd3c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:53.651904 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992cbd68-94eb-460d-bb25-79dc70dfd3c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:53.653423 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992cbd68-94eb-460d-bb25-79dc70dfd3c9 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:53.661306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ba12e-7c55-4c74-bfd2-72f7c46fbb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:53.661943 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ba12e-7c55-4c74-bfd2-72f7c46fbb60 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:53.662221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ba12e-7c55-4c74-bfd2-72f7c46fbb60 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:53.662493 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ba12e-7c55-4c74-bfd2-72f7c46fbb60 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:53.676166 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ba12e-7c55-4c74-bfd2-72f7c46fbb60 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:53.676166 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ba12e-7c55-4c74-bfd2-72f7c46fbb60 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:53.676426 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 321/623] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:53.677307 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1c4da-8733-4f94-ad88-a2ff807d93c1 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:53.677592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1c4da-8733-4f94-ad88-a2ff807d93c1 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:53.677863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1c4da-8733-4f94-ad88-a2ff807d93c1 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:53.679282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 322/624] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:53.680691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1c4da-8733-4f94-ad88-a2ff807d93c1 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:53.695845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-a86ba12e-7c55-4c74-bfd2-72f7c46fbb60 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:59:53.695845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:53.695845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:53.695845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:53.695845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:53.695845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:53.695845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:53.695845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:53.695845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:53.695845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:53.695845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:53.695845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:53.695845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:59:53.695845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:59:53.695845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:59:53.695845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:59:53.696808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:53.696808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:53.696808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:59:53.696808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:53.696808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:59:53.696808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:53.696808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:53.696808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:53.696808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:53.696808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:53.696808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:53.696808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:53.696808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:53.696808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:53.696808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:53.696808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:53.696808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:53.696808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:53.697737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:53.697737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:53.697737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:59:53.697737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:59:53.697737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:59:53.697737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:59:53.697737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:59:53.697737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:59:53.697737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:59:53.697737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:53.700387 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 303/625] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Aug 23 06:59:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:59:53.702250 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebde73-5492-489e-997a-32831edc1288 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:53.702726 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebde73-5492-489e-997a-32831edc1288 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:53.703325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebde73-5492-489e-997a-32831edc1288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:53.723598 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-7bebde73-5492-489e-997a-32831edc1288 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:53.725165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-7bebde73-5492-489e-997a-32831edc1288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:53.807522 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 304/626] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:53.808016 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:53.808433 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebde73-5492-489e-997a-32831edc1288 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:53.811911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1c4da-8733-4f94-ad88-a2ff807d93c1 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:53.871766 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb51f7b-0a65-48e9-8c28-8c07179af1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:53.872843 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb51f7b-0a65-48e9-8c28-8c07179af1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:53.873310 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb51f7b-0a65-48e9-8c28-8c07179af1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:53.873798 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb51f7b-0a65-48e9-8c28-8c07179af1a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:53.882273 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb51f7b-0a65-48e9-8c28-8c07179af1a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:53.884482 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb51f7b-0a65-48e9-8c28-8c07179af1a2 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:53.896974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-1fb51f7b-0a65-48e9-8c28-8c07179af1a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:53.896974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:53.896974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:53.896974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:53.896974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:53.896974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:53.896974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:53.896974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:53.896974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:53.896974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:53.896974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:53.896974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:53.896974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:59:53.896974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:59:53.896974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:59:53.897886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:59:53.897886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:53.897886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:53.897886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:53.897886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:53.897886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:53.897886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:53.897886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:53.897886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:53.897886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:53.897886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:53.897886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:53.897886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:53.897886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:53.897886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:53.897886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:53.897886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:53.898761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:59:53.898761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:59:53.898761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:59:53.898761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:59:53.898761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:59:53.898761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:59:53.898761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:53.898761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:53.906223 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 323/627] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:53.907699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb51f7b-0a65-48e9-8c28-8c07179af1a2 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:54.020345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebde73-5492-489e-997a-32831edc1288 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:54.043766 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:54.086366 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb51f7b-0a65-48e9-8c28-8c07179af1a2 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:54.088842 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ffec8-3c44-4cf3-9f72-1c071ed04395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:54.089463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ffec8-3c44-4cf3-9f72-1c071ed04395 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:54.089732 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ffec8-3c44-4cf3-9f72-1c071ed04395 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:54.089996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ffec8-3c44-4cf3-9f72-1c071ed04395 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:54.090730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ffec8-3c44-4cf3-9f72-1c071ed04395 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:54.091027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ffec8-3c44-4cf3-9f72-1c071ed04395 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:54.092497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ffec8-3c44-4cf3-9f72-1c071ed04395 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:54.120966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 305/628] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Aug 23 06:59:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:54.123685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce65996-ae03-4d96-9a77-3b5d4849ee99 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:54.124028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce65996-ae03-4d96-9a77-3b5d4849ee99 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:54.124364 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce65996-ae03-4d96-9a77-3b5d4849ee99 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:54.126030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 306/629] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:59:54.126913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce65996-ae03-4d96-9a77-3b5d4849ee99 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:54.149684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceed866-ae2a-417d-ac28-9a5e76786a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:54.150635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceed866-ae2a-417d-ac28-9a5e76786a28 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:54.151059 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceed866-ae2a-417d-ac28-9a5e76786a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:54.151503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceed866-ae2a-417d-ac28-9a5e76786a28 None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:54.152340 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceed866-ae2a-417d-ac28-9a5e76786a28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:54.152925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceed866-ae2a-417d-ac28-9a5e76786a28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:54.154630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceed866-ae2a-417d-ac28-9a5e76786a28 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:54.167400 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 324/630] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 06:59:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:59:54.168819 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf01464-b356-434b-9e46-cba39ca98d00 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:54.169251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf01464-b356-434b-9e46-cba39ca98d00 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:54.169696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf01464-b356-434b-9e46-cba39ca98d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:54.188256 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-faf01464-b356-434b-9e46-cba39ca98d00 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:54.189780 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-faf01464-b356-434b-9e46-cba39ca98d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:54.205927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:54.235715 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce65996-ae03-4d96-9a77-3b5d4849ee99 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:54.268976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 325/631] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:54.268976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf01464-b356-434b-9e46-cba39ca98d00 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:54.273802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcebcbd5-4b8a-4825-b8c7-f812ad1df585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:54.275395 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebcbd5-4b8a-4825-b8c7-f812ad1df585 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:54.275704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebcbd5-4b8a-4825-b8c7-f812ad1df585 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:54.276401 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebcbd5-4b8a-4825-b8c7-f812ad1df585 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:54.332031 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:54.359919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcebcbd5-4b8a-4825-b8c7-f812ad1df585 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:54.361347 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcebcbd5-4b8a-4825-b8c7-f812ad1df585 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:54.393559 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf01464-b356-434b-9e46-cba39ca98d00 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:54.430463 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4298c22-5322-474a-b07d-6bbf72e11335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:54.431501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4298c22-5322-474a-b07d-6bbf72e11335 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:54.431980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4298c22-5322-474a-b07d-6bbf72e11335 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:54.432443 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4298c22-5322-474a-b07d-6bbf72e11335 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:54.443064 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4298c22-5322-474a-b07d-6bbf72e11335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:54.448255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-f4298c22-5322-474a-b07d-6bbf72e11335 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:54.448255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:54.448255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:54.448255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:54.448255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:54.448255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:54.448255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:54.448255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:54.448255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:54.448255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:54.448255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:54.448255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:54.448255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:54.448255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:54.448255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:54.448255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:54.449168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:54.449168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:54.449168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:54.449168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:54.449168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:54.449168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:54.449168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:54.449168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:54.449168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:54.449168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:54.449168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:54.449168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:54.449168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:54.449168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:54.449168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:54.449168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:54.450025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:54.450025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:54.450025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:54.450025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:54.450025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:54.450025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:54.450025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:54.450025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:54.450025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:54.450025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:54.450025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:54.450025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:54.457611 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 326/632] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 06:59:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:54.458693 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4298c22-5322-474a-b07d-6bbf72e11335 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:54.471642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:54.481887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 307/633] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:54.481887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcebcbd5-4b8a-4825-b8c7-f812ad1df585 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:54.553246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:54.568775 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4298c22-5322-474a-b07d-6bbf72e11335 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:54.593641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcebcbd5-4b8a-4825-b8c7-f812ad1df585 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:54.607060 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2f9ea-84f8-4fb0-ac05-8b17b6cd7d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:54.607750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2f9ea-84f8-4fb0-ac05-8b17b6cd7d1a None admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:54.608030 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2f9ea-84f8-4fb0-ac05-8b17b6cd7d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:54.609049 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2f9ea-84f8-4fb0-ac05-8b17b6cd7d1a None admin] PATH_INFO: `/v3/projects/351a6777525c4207826d7df0d3768e99/users/fd88ec8b477443af95bff9af03471adf/roles/2d62b8b8395745c7983c6c8d100d985b` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:54.624169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2f9ea-84f8-4fb0-ac05-8b17b6cd7d1a None admin] RBAC: Authorizing `identity:create_grant(project_id=351a6777525c4207826d7df0d3768e99, user_id=fd88ec8b477443af95bff9af03471adf, role_id=2d62b8b8395745c7983c6c8d100d985b)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:54.625615 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2f9ea-84f8-4fb0-ac05-8b17b6cd7d1a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:54.629297 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2192b-fd6d-4449-b80a-b9b6c5a98603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:54.629912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2192b-fd6d-4449-b80a-b9b6c5a98603 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:54.630190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2192b-fd6d-4449-b80a-b9b6c5a98603 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:54.630463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2192b-fd6d-4449-b80a-b9b6c5a98603 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:54.631128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2192b-fd6d-4449-b80a-b9b6c5a98603 None admin] RBAC: Adding query filter params (name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:54.631513 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2192b-fd6d-4449-b80a-b9b6c5a98603 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:54.632922 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2192b-fd6d-4449-b80a-b9b6c5a98603 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:54.639468 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 327/634] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 06:59:54 2022] PUT /identity/v3/projects/351a6777525c4207826d7df0d3768e99/users/fd88ec8b477443af95bff9af03471adf/roles/2d62b8b8395745c7983c6c8d100d985b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:59:54.640198 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2f9ea-84f8-4fb0-ac05-8b17b6cd7d1a None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:54.653757 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 308/635] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 06:59:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:54.654632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2192b-fd6d-4449-b80a-b9b6c5a98603 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:54.720071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:54.768915 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2f9ea-84f8-4fb0-ac05-8b17b6cd7d1a None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:54.774670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2192b-fd6d-4449-b80a-b9b6c5a98603 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:54.808396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:54.814558 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676b55e-6101-4c0e-991c-ce550731a103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:54.815199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676b55e-6101-4c0e-991c-ce550731a103 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:54.815625 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676b55e-6101-4c0e-991c-ce550731a103 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:54.815907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676b55e-6101-4c0e-991c-ce550731a103 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:54.817129 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a257c-e8db-4c3f-8461-0000677aa067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:54.817839 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a257c-e8db-4c3f-8461-0000677aa067 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:54.818191 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a257c-e8db-4c3f-8461-0000677aa067 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:54.818372 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a257c-e8db-4c3f-8461-0000677aa067 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:54.827538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676b55e-6101-4c0e-991c-ce550731a103 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:54.829044 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676b55e-6101-4c0e-991c-ce550731a103 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:54.829727 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a257c-e8db-4c3f-8461-0000677aa067 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:54.835441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-4e8a257c-e8db-4c3f-8461-0000677aa067 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:54.835441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:54.835441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:54.835441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:54.835441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:54.835441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:54.835441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:54.835441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:54.835441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:54.835441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:54.835441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:54.835441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:54.835441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:54.835441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:54.835441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:54.835441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:54.836539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:54.836539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:54.836539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:54.836539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:54.836539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:54.836539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:54.836539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:54.836539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:54.836539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:54.836539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:54.836539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:54.836539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:54.836539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:54.836539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:54.836539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:54.836539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:54.837470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:54.837470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:54.837470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:54.837470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:54.837470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:54.837470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:54.837470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:54.837470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:54.837470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:54.837470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:54.837470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:54.837470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:54.838446 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 309/636] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 06:59:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:54.843702 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b927a-637e-4e66-ac6c-1bef1eae5d82 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:54.843702 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b927a-637e-4e66-ac6c-1bef1eae5d82 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:54.843702 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b927a-637e-4e66-ac6c-1bef1eae5d82 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:54.843702 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 310/637] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:54.843702 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b927a-637e-4e66-ac6c-1bef1eae5d82 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:54.845395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-c676b55e-6101-4c0e-991c-ce550731a103 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:54.845395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:54.845395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:54.845395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:54.845395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:54.845395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:54.845395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:54.845395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:54.845395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:54.845395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:54.845395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:54.845395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:54.845395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:54.845395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:54.845395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:54.845395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:54.846339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:54.846339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:54.846339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:54.846339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:54.846339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:54.846339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:54.846339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:54.846339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:54.846339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:54.846339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:54.846339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:54.846339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:54.846339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:54.846339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:54.846339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:54.846339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:54.846339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:54.847324 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:54.847324 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:54.847324 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:54.847324 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 328/638] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:54.851720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676b55e-6101-4c0e-991c-ce550731a103 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:54.955497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b927a-637e-4e66-ac6c-1bef1eae5d82 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:54.956472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676b55e-6101-4c0e-991c-ce550731a103 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:54.969565 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:54.991041 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f17da-6f47-4015-8b0c-e1458d5527b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:54.991865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f17da-6f47-4015-8b0c-e1458d5527b5 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:54.992279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f17da-6f47-4015-8b0c-e1458d5527b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:54.992609 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f17da-6f47-4015-8b0c-e1458d5527b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:54.992899 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68dc8da-adb4-43f4-aefa-63e876dcfaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:54.993516 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68dc8da-adb4-43f4-aefa-63e876dcfaa3 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:54.993780 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68dc8da-adb4-43f4-aefa-63e876dcfaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:54.994050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68dc8da-adb4-43f4-aefa-63e876dcfaa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:54.994706 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68dc8da-adb4-43f4-aefa-63e876dcfaa3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:54.995158 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68dc8da-adb4-43f4-aefa-63e876dcfaa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:54.996660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68dc8da-adb4-43f4-aefa-63e876dcfaa3 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:55.004335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f17da-6f47-4015-8b0c-e1458d5527b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:55.009704 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 329/639] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 06:59:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:55.009851 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f17da-6f47-4015-8b0c-e1458d5527b5 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:55.009940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f51471-2898-4f90-a00d-4a3e04cadeb1 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:55.009940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f51471-2898-4f90-a00d-4a3e04cadeb1 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:55.009940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f51471-2898-4f90-a00d-4a3e04cadeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:55.025499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-871f17da-6f47-4015-8b0c-e1458d5527b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:55.025499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:55.025499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:55.025499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:55.025499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:55.025499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:55.025499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:55.025499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:55.025499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:55.025499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:55.025499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:55.025499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:55.025499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:55.025499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:55.025499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:55.025499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:55.026500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:55.026500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:55.026500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:55.026500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:55.026500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:55.026500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:55.026500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:55.026500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:55.026500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:55.026500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:55.026500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:55.026500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:55.026500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:55.026500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:55.026500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:55.026500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:55.026500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:55.027400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:55.027400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:55.027400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:55.027600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-75f51471-2898-4f90-a00d-4a3e04cadeb1 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:55.027600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-75f51471-2898-4f90-a00d-4a3e04cadeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:55.031392 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 311/640] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:55.032289 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f17da-6f47-4015-8b0c-e1458d5527b5 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:55.077468 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:55.100228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 330/641] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:55.101303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f51471-2898-4f90-a00d-4a3e04cadeb1 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:55.141280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f17da-6f47-4015-8b0c-e1458d5527b5 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:55.178542 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7047327-e078-451f-88d2-d2f0354c59a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:55.179488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7047327-e078-451f-88d2-d2f0354c59a4 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:55.179961 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7047327-e078-451f-88d2-d2f0354c59a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:55.180407 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7047327-e078-451f-88d2-d2f0354c59a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:55.181307 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7047327-e078-451f-88d2-d2f0354c59a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:55.181782 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7047327-e078-451f-88d2-d2f0354c59a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:55.183387 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7047327-e078-451f-88d2-d2f0354c59a4 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:55.201089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 312/642] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:55.202511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7047327-e078-451f-88d2-d2f0354c59a4 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:55.232939 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f51471-2898-4f90-a00d-4a3e04cadeb1 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:55.236274 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:55.279996 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61919b32-442b-479d-9aca-766f3e3c79bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:55.280661 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61919b32-442b-479d-9aca-766f3e3c79bd None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:55.280941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61919b32-442b-479d-9aca-766f3e3c79bd None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:55.281209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61919b32-442b-479d-9aca-766f3e3c79bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:55.294261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61919b32-442b-479d-9aca-766f3e3c79bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:55.296367 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61919b32-442b-479d-9aca-766f3e3c79bd None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:55.312840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-61919b32-442b-479d-9aca-766f3e3c79bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:55.312840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:55.312840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:55.312840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:55.312840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:55.312840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:55.312840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:55.312840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:55.312840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:55.312840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:55.312840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:55.312840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:55.312840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:55.312840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:55.312840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:55.312840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:55.313891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:55.313891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:55.313891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:55.313891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:55.313891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:55.313891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:55.313891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:55.313891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:55.313891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:55.313891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:55.313891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:55.313891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:55.313891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:55.313891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:55.313891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:55.313891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:55.313891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:55.314811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:55.314811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:55.314811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:55.315474 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7047327-e078-451f-88d2-d2f0354c59a4 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:55.316878 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 331/643] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:55.317804 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61919b32-442b-479d-9aca-766f3e3c79bd None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:55.321930 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:55.364473 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75002360-e426-4ab6-85d7-96ca2f28851c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:55.365041 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75002360-e426-4ab6-85d7-96ca2f28851c None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:55.365318 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75002360-e426-4ab6-85d7-96ca2f28851c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:55.365591 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75002360-e426-4ab6-85d7-96ca2f28851c None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:55.366509 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75002360-e426-4ab6-85d7-96ca2f28851c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:55.366811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75002360-e426-4ab6-85d7-96ca2f28851c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:55.368385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75002360-e426-4ab6-85d7-96ca2f28851c None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:55.381369 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 313/644] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:55.382180 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75002360-e426-4ab6-85d7-96ca2f28851c None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:55.429457 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61919b32-442b-479d-9aca-766f3e3c79bd None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:55.467792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027585b9-1e9a-4174-a952-bc092330de90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:55.468424 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027585b9-1e9a-4174-a952-bc092330de90 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:55.468700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027585b9-1e9a-4174-a952-bc092330de90 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:55.468975 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027585b9-1e9a-4174-a952-bc092330de90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:55.476040 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027585b9-1e9a-4174-a952-bc092330de90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:55.477504 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027585b9-1e9a-4174-a952-bc092330de90 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:55.484454 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75002360-e426-4ab6-85d7-96ca2f28851c None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:55.488941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-027585b9-1e9a-4174-a952-bc092330de90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:55.488941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:55.488941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:55.488941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:55.488941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:55.488941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:55.488941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:55.488941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:55.488941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:55.488941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:55.488941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:55.488941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:55.488941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:59:55.488941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:59:55.488941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:59:55.489814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:59:55.489814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:55.489814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:55.489814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:55.489814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:55.489814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:55.489814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:55.489814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:55.489814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:55.489814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:55.489814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:55.489814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:55.489814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:55.489814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:55.489814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:55.489814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:55.489814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:55.490866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:59:55.490866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:59:55.490866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:59:55.490866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:59:55.490866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:59:55.490866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:59:55.490866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:55.490866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:55.494768 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 332/645] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:55.495669 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027585b9-1e9a-4174-a952-bc092330de90 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:55.515047 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:55.520940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61e05d-e441-4a38-8a19-0100e2106a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:55.521544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61e05d-e441-4a38-8a19-0100e2106a85 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:55.521814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61e05d-e441-4a38-8a19-0100e2106a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:55.522085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61e05d-e441-4a38-8a19-0100e2106a85 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:55.522735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61e05d-e441-4a38-8a19-0100e2106a85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:55.523086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61e05d-e441-4a38-8a19-0100e2106a85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:55.524556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61e05d-e441-4a38-8a19-0100e2106a85 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:55.542227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 314/646] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:55.542227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61e05d-e441-4a38-8a19-0100e2106a85 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:55.566435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:55.607936 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027585b9-1e9a-4174-a952-bc092330de90 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:55.646746 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61e05d-e441-4a38-8a19-0100e2106a85 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:55.654699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a64472-29a5-4ec2-81ca-d617d4daa06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:55.655016 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a64472-29a5-4ec2-81ca-d617d4daa06a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:55.655243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a64472-29a5-4ec2-81ca-d617d4daa06a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:55.655541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a64472-29a5-4ec2-81ca-d617d4daa06a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:55.667036 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a64472-29a5-4ec2-81ca-d617d4daa06a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:55.672622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-b4a64472-29a5-4ec2-81ca-d617d4daa06a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:55.672622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:55.672622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:55.672622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:55.672622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:55.672622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:55.672622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:55.672622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:55.672622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:55.672622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:55.672622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:55.672622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:55.672622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:55.672622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:55.672622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:55.672622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:55.673537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:55.673537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:55.673537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:55.673537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:55.673537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:55.673537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:55.673537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:55.673537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:55.673537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:55.673537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:55.673537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:55.673537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:55.673537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:55.673537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:55.673537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:55.673537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:55.674381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:55.674381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:55.674381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:55.674381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:55.674381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:55.674381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:55.674381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:55.674381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:55.674381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:55.674381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:55.674381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:55.674381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:55.674381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 333/647] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 06:59:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:55.675856 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a64472-29a5-4ec2-81ca-d617d4daa06a None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:55.690075 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659f5c4-f11a-4861-ae9d-9b99126dfbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:55.690770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659f5c4-f11a-4861-ae9d-9b99126dfbc2 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:55.691052 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659f5c4-f11a-4861-ae9d-9b99126dfbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:55.691328 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659f5c4-f11a-4861-ae9d-9b99126dfbc2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:55.699651 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e659f5c4-f11a-4861-ae9d-9b99126dfbc2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:55.701148 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e659f5c4-f11a-4861-ae9d-9b99126dfbc2 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:55.714729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-e659f5c4-f11a-4861-ae9d-9b99126dfbc2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 06:59:55.714729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:55.714729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:55.714729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:55.714729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:55.714729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:55.714729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:55.714729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:55.714729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:55.714729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:55.714729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:55.714729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:55.714729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:59:55.714729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:59:55.714729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:59:55.714729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:59:55.715781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:55.715781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:55.715781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:59:55.715781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:55.715781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:59:55.715781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:55.715781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:55.715781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:55.715781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:55.715781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:55.715781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:55.715781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:55.715781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:55.715781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:55.715781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:55.715781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:55.715781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:55.715781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:55.716728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:55.716728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:55.716728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:59:55.716728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:59:55.716728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:59:55.716728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:59:55.716728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:59:55.716728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:59:55.716728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 06:59:55.716728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:55.720649 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 315/648] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:55.721497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659f5c4-f11a-4861-ae9d-9b99126dfbc2 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:55.759120 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:55.782066 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a64472-29a5-4ec2-81ca-d617d4daa06a None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:55.812194 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:55.817520 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5aec8-04cb-4d7c-83d0-6c9ce050090d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:55.818115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5aec8-04cb-4d7c-83d0-6c9ce050090d None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:55.818390 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5aec8-04cb-4d7c-83d0-6c9ce050090d None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:55.818664 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5aec8-04cb-4d7c-83d0-6c9ce050090d None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:55.819353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5aec8-04cb-4d7c-83d0-6c9ce050090d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:55.819689 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5aec8-04cb-4d7c-83d0-6c9ce050090d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:55.821126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5aec8-04cb-4d7c-83d0-6c9ce050090d None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:55.832463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659f5c4-f11a-4861-ae9d-9b99126dfbc2 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:55.838653 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 334/649] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 06:59:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:55.838653 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5aec8-04cb-4d7c-83d0-6c9ce050090d None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:55.872533 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e974f6-5e6a-4ad3-a27f-37e0800b7850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:55.873156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e974f6-5e6a-4ad3-a27f-37e0800b7850 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:55.873430 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e974f6-5e6a-4ad3-a27f-37e0800b7850 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:55.873697 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e974f6-5e6a-4ad3-a27f-37e0800b7850 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:55.881505 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e974f6-5e6a-4ad3-a27f-37e0800b7850 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:55.883060 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e974f6-5e6a-4ad3-a27f-37e0800b7850 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:55.895086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-86e974f6-5e6a-4ad3-a27f-37e0800b7850 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 06:59:55.895086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:55.895086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:55.895086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:55.895086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:55.895086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:55.895086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:55.895086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:55.895086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:55.895086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:55.895086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:55.895086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:55.895086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:59:55.895086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:59:55.895086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:59:55.895086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:59:55.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:55.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:55.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:59:55.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:55.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:59:55.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:55.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:55.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:55.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:55.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:55.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:55.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:55.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:55.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:55.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:55.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:55.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:55.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:55.897030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:55.897030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:55.897030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:59:55.897030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:59:55.897030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:59:55.897030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:59:55.897030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:59:55.897030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:59:55.897030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 06:59:55.897030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:55.897030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 316/650] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:55.897030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e974f6-5e6a-4ad3-a27f-37e0800b7850 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:55.940956 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5aec8-04cb-4d7c-83d0-6c9ce050090d None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:55.978423 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95088c-8e26-4fa5-b35b-e3f61f35d025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:55.979031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95088c-8e26-4fa5-b35b-e3f61f35d025 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:55.979308 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95088c-8e26-4fa5-b35b-e3f61f35d025 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:55.979607 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95088c-8e26-4fa5-b35b-e3f61f35d025 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:55.980297 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95088c-8e26-4fa5-b35b-e3f61f35d025 None admin] RBAC: Adding query filter params (name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:55.980683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95088c-8e26-4fa5-b35b-e3f61f35d025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:55.981936 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95088c-8e26-4fa5-b35b-e3f61f35d025 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:55.984370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:55.994408 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 335/651] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 06:59:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:55.995539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95088c-8e26-4fa5-b35b-e3f61f35d025 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:56.009234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e974f6-5e6a-4ad3-a27f-37e0800b7850 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:56.045073 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787d161-b58a-4baa-8198-0931f08b86e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:56.045840 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787d161-b58a-4baa-8198-0931f08b86e1 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:56.045951 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787d161-b58a-4baa-8198-0931f08b86e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:56.046218 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787d161-b58a-4baa-8198-0931f08b86e1 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:56.046945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787d161-b58a-4baa-8198-0931f08b86e1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:56.047244 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787d161-b58a-4baa-8198-0931f08b86e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:56.048727 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787d161-b58a-4baa-8198-0931f08b86e1 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:56.064043 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:56.072898 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 317/652] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Aug 23 06:59:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:56.073594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787d161-b58a-4baa-8198-0931f08b86e1 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:56.101998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95088c-8e26-4fa5-b35b-e3f61f35d025 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:56.135835 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b3c71-ee34-434a-bd24-ab15dc7a9cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:56.136435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b3c71-ee34-434a-bd24-ab15dc7a9cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:56.136722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b3c71-ee34-434a-bd24-ab15dc7a9cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:56.136974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b3c71-ee34-434a-bd24-ab15dc7a9cf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:56.137691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b3c71-ee34-434a-bd24-ab15dc7a9cf9 None admin] RBAC: Adding query filter params (role.id=2d62b8b8395745c7983c6c8d100d985b, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=a8d2cd93042c4583b63afe629d47fce0) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:56.137989 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b3c71-ee34-434a-bd24-ab15dc7a9cf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:56.139434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b3c71-ee34-434a-bd24-ab15dc7a9cf9 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:56.145300 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 336/653] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:59:55 2022] GET /identity/v3/role_assignments?user.id=a8d2cd93042c4583b63afe629d47fce0&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2d62b8b8395745c7983c6c8d100d985b => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:56.146084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b3c71-ee34-434a-bd24-ab15dc7a9cf9 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:56.201146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787d161-b58a-4baa-8198-0931f08b86e1 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:56.218400 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:56.242959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0a2ac-faf0-42c8-9f91-5740dd0b8c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:56.243967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0a2ac-faf0-42c8-9f91-5740dd0b8c22 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:56.244448 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0a2ac-faf0-42c8-9f91-5740dd0b8c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:56.244896 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0a2ac-faf0-42c8-9f91-5740dd0b8c22 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:56.245819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0a2ac-faf0-42c8-9f91-5740dd0b8c22 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:56.246312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0a2ac-faf0-42c8-9f91-5740dd0b8c22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:56.248187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0a2ac-faf0-42c8-9f91-5740dd0b8c22 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:56.263056 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b3c71-ee34-434a-bd24-ab15dc7a9cf9 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:56.272960 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 318/654] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Aug 23 06:59:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:56.274233 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0a2ac-faf0-42c8-9f91-5740dd0b8c22 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:56.303465 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a7e39a-f595-429f-ad44-bda54eaaae4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:56.304101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7e39a-f595-429f-ad44-bda54eaaae4f None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:56.304376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7e39a-f595-429f-ad44-bda54eaaae4f None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:56.304707 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7e39a-f595-429f-ad44-bda54eaaae4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:56.320925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a7e39a-f595-429f-ad44-bda54eaaae4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:56.323296 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a7e39a-f595-429f-ad44-bda54eaaae4f None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:56.345814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-33a7e39a-f595-429f-ad44-bda54eaaae4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:56.345814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:56.345814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:56.345814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:56.345814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:56.345814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:56.345814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:56.345814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:56.345814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:56.345814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:56.345814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:56.345814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:56.345814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:56.345814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:56.345814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:56.345814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:56.347029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:56.347029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:56.347029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:56.347029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:56.347029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:56.347029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:56.347029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:56.347029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:56.347029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:56.347029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:56.347029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:56.347029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:56.347029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:56.347029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:56.347029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:56.347029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:56.347029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:56.348419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:56.348419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:56.348419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:56.348419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 337/655] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:56.348688 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a7e39a-f595-429f-ad44-bda54eaaae4f None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:56.350998 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:56.411681 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0a2ac-faf0-42c8-9f91-5740dd0b8c22 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:56.458144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669c17d-de8d-4447-a47b-fc999c31a721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:56.459126 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669c17d-de8d-4447-a47b-fc999c31a721 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:56.459645 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669c17d-de8d-4447-a47b-fc999c31a721 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:56.460123 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669c17d-de8d-4447-a47b-fc999c31a721 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:56.476011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a7e39a-f595-429f-ad44-bda54eaaae4f None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:56.491969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:56.516882 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b25fb-cb06-409c-b170-65c58f09dcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:56.517542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b25fb-cb06-409c-b170-65c58f09dcf6 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:56.517824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b25fb-cb06-409c-b170-65c58f09dcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:56.518101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b25fb-cb06-409c-b170-65c58f09dcf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:56.546541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669c17d-de8d-4447-a47b-fc999c31a721 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:56.548852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669c17d-de8d-4447-a47b-fc999c31a721 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:56.590933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b25fb-cb06-409c-b170-65c58f09dcf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:56.592426 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b25fb-cb06-409c-b170-65c58f09dcf6 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:56.612651 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:56.662148 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 319/656] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:56.663014 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669c17d-de8d-4447-a47b-fc999c31a721 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:56.717793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 338/657] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:56.719285 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed848e-629b-4147-8f90-e13941fd17eb None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:56.719619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed848e-629b-4147-8f90-e13941fd17eb None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:56.719897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed848e-629b-4147-8f90-e13941fd17eb None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:56.723469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 339/658] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:56.723469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed848e-629b-4147-8f90-e13941fd17eb None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:56.739222 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:56.781090 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669c17d-de8d-4447-a47b-fc999c31a721 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:56.818376 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527cf32-65d5-4304-bcf6-1d1299153a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:56.819164 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527cf32-65d5-4304-bcf6-1d1299153a06 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:56.819450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527cf32-65d5-4304-bcf6-1d1299153a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:56.819742 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527cf32-65d5-4304-bcf6-1d1299153a06 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:56.820400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527cf32-65d5-4304-bcf6-1d1299153a06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:56.820705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527cf32-65d5-4304-bcf6-1d1299153a06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:56.822089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527cf32-65d5-4304-bcf6-1d1299153a06 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:56.834232 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 320/659] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:56.834958 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed848e-629b-4147-8f90-e13941fd17eb None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:56.835072 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527cf32-65d5-4304-bcf6-1d1299153a06 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:56.857258 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:56.878035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953ceef-4d74-42dc-a5a5-03f5346bea07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:56.878752 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953ceef-4d74-42dc-a5a5-03f5346bea07 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:56.879031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953ceef-4d74-42dc-a5a5-03f5346bea07 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:56.879298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953ceef-4d74-42dc-a5a5-03f5346bea07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:56.890167 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953ceef-4d74-42dc-a5a5-03f5346bea07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:56.891649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953ceef-4d74-42dc-a5a5-03f5346bea07 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:56.907535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-a953ceef-4d74-42dc-a5a5-03f5346bea07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:56.907535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:56.907535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:56.907535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:56.907535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:56.907535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:56.907535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:56.907535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:56.907535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:56.907535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:56.907535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:56.907535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:56.907535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:56.907535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:56.907535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:56.907535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:56.908694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:56.908694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:56.908694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:56.908694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:56.908694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:56.908694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:56.908694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:56.908694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:56.908694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:56.908694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:56.908694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:56.908694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:56.908694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:56.908694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:56.908694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:56.908694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:56.908694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:56.909755 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:56.909755 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:56.909755 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:56.909755 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 340/660] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:56.910456 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ade9fb-da31-42ef-9e2d-8da9993cb42f None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:56.910771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ade9fb-da31-42ef-9e2d-8da9993cb42f None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:56.911010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ade9fb-da31-42ef-9e2d-8da9993cb42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:56.930982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-92ade9fb-da31-42ef-9e2d-8da9993cb42f None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:56.932354 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-92ade9fb-da31-42ef-9e2d-8da9993cb42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:56.947910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527cf32-65d5-4304-bcf6-1d1299153a06 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:56.991972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c771a-f1f2-4797-855e-9e15ff62b8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:56.992601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c771a-f1f2-4797-855e-9e15ff62b8fd None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:56.992795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c771a-f1f2-4797-855e-9e15ff62b8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:56.993042 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c771a-f1f2-4797-855e-9e15ff62b8fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:57.002373 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:57.009219 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c771a-f1f2-4797-855e-9e15ff62b8fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:57.010740 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c771a-f1f2-4797-855e-9e15ff62b8fd None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:57.015637 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 341/661] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:57.015637 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ade9fb-da31-42ef-9e2d-8da9993cb42f None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:57.034878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-d33c771a-f1f2-4797-855e-9e15ff62b8fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:57.034878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:57.034878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:57.034878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:57.034878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:57.034878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:57.034878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:57.034878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:57.034878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:57.034878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:57.034878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:57.034878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:57.034878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:57.034878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:57.034878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:57.034878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:57.036034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:57.036034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:57.036034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:57.036034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:57.036034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:57.036034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:57.036034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:57.036034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:57.036034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:57.036034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:57.036034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:57.036034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:57.036034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:57.036034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:57.036034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:57.036034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:57.036034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:57.037016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:57.037016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:57.037016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:57.037016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 321/662] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:57.037016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c771a-f1f2-4797-855e-9e15ff62b8fd None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:57.115617 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:57.148651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ade9fb-da31-42ef-9e2d-8da9993cb42f None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:57.148855 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c771a-f1f2-4797-855e-9e15ff62b8fd None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:57.192226 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ef009-f868-4800-b61e-76fb9ec4196a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:57.193249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ef009-f868-4800-b61e-76fb9ec4196a None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:57.193369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d377d-b86e-4d76-8663-8ea5f6e7ccc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:57.193875 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ef009-f868-4800-b61e-76fb9ec4196a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:57.193991 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d377d-b86e-4d76-8663-8ea5f6e7ccc1 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:57.194231 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d377d-b86e-4d76-8663-8ea5f6e7ccc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:57.194500 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d377d-b86e-4d76-8663-8ea5f6e7ccc1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:57.194868 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ef009-f868-4800-b61e-76fb9ec4196a None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:57.195735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ef009-f868-4800-b61e-76fb9ec4196a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:57.196228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ef009-f868-4800-b61e-76fb9ec4196a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:57.198503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ef009-f868-4800-b61e-76fb9ec4196a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:57.203380 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d377d-b86e-4d76-8663-8ea5f6e7ccc1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:57.204965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d377d-b86e-4d76-8663-8ea5f6e7ccc1 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:57.213728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 322/663] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:57.215174 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ef009-f868-4800-b61e-76fb9ec4196a None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:57.224883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-727d377d-b86e-4d76-8663-8ea5f6e7ccc1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 06:59:57.224883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:57.224883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:57.224883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:57.224883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:57.224883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:57.224883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:57.224883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:57.224883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:57.224883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:57.224883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:57.224883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:57.224883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:59:57.224883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:59:57.224883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:59:57.224883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:59:57.225907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:57.225907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:57.225907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:59:57.225907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:57.225907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:59:57.225907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:57.225907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:57.225907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:57.225907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:57.225907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:57.225907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:57.225907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:57.225907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:57.225907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:57.225907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:57.225907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:57.225907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:57.225907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:57.226895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:57.226895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:57.226895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:59:57.226895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:59:57.226895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:59:57.226895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:59:57.226895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:59:57.226895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:59:57.226895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 06:59:57.226895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:57.226895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 342/664] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 06:59:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:57.227584 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d377d-b86e-4d76-8663-8ea5f6e7ccc1 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:57.297977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:57.331193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ef009-f868-4800-b61e-76fb9ec4196a None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:57.345197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d377d-b86e-4d76-8663-8ea5f6e7ccc1 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:57.362615 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:57.369888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58966f-7b00-4e8d-be28-0f08c5540e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:57.370595 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58966f-7b00-4e8d-be28-0f08c5540e81 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:57.370869 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58966f-7b00-4e8d-be28-0f08c5540e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:57.371122 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58966f-7b00-4e8d-be28-0f08c5540e81 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:57.371806 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58966f-7b00-4e8d-be28-0f08c5540e81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:57.372104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58966f-7b00-4e8d-be28-0f08c5540e81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:57.373839 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58966f-7b00-4e8d-be28-0f08c5540e81 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:57.381473 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b46365-4362-44fd-a644-b5ec1f33b00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:57.382124 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b46365-4362-44fd-a644-b5ec1f33b00e None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:57.382407 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b46365-4362-44fd-a644-b5ec1f33b00e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:57.382673 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b46365-4362-44fd-a644-b5ec1f33b00e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:57.390362 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b46365-4362-44fd-a644-b5ec1f33b00e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:57.390811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 323/665] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:57.391006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b46365-4362-44fd-a644-b5ec1f33b00e None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:57.391625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58966f-7b00-4e8d-be28-0f08c5540e81 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:57.405570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-96b46365-4362-44fd-a644-b5ec1f33b00e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:57.405570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:57.405570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:57.405570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:57.405570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:57.405570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:57.405570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:57.405570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:57.405570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:57.405570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:57.405570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:57.405570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:57.405570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:59:57.405570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:59:57.405570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:59:57.406487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:59:57.406487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:57.406487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:57.406487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:57.406487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:57.406487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:57.406487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:57.406487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:57.406487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:57.406487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:57.406487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:57.406487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:57.406487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:57.406487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:57.406487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:57.406487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:57.406487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:57.412556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:59:57.412556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:59:57.412556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:59:57.412556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:59:57.412556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:59:57.412556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:59:57.412556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:57.412556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:57.412556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 343/666] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:57.412556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b46365-4362-44fd-a644-b5ec1f33b00e None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:57.513847 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b46365-4362-44fd-a644-b5ec1f33b00e None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:57.514510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58966f-7b00-4e8d-be28-0f08c5540e81 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:57.536151 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:57.550288 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba2117-a362-4c98-bb79-e625d928e048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:57.551059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba2117-a362-4c98-bb79-e625d928e048 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:57.551445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba2117-a362-4c98-bb79-e625d928e048 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:57.551746 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba2117-a362-4c98-bb79-e625d928e048 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:57.552457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba2117-a362-4c98-bb79-e625d928e048 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:57.552769 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba2117-a362-4c98-bb79-e625d928e048 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:57.554207 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba2117-a362-4c98-bb79-e625d928e048 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:57.557063 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff29ef2-bdb0-487a-a833-0fd0e86187c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:57.557689 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff29ef2-bdb0-487a-a833-0fd0e86187c3 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:57.557964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff29ef2-bdb0-487a-a833-0fd0e86187c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:57.558229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff29ef2-bdb0-487a-a833-0fd0e86187c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:57.569488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff29ef2-bdb0-487a-a833-0fd0e86187c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:57.577293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 324/667] 10.222.0.17 () {56 vars in 1139 bytes} [Tue Aug 23 06:59:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:57.577293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba2117-a362-4c98-bb79-e625d928e048 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:57.577522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-3ff29ef2-bdb0-487a-a833-0fd0e86187c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:57.577522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:57.577522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:57.577522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:57.577522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:57.577522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:57.577522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:57.577522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:57.577522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:57.577522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:57.577522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:57.577522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:57.577522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:57.577522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:57.577522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:57.577522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:57.578409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:57.578409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:57.578409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:57.578409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:57.578409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:57.578409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:57.578409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:57.578409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:57.578409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:57.578409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:57.578409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:57.578409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:57.578409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:57.578409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:57.578409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:57.578409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:57.579255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:57.579255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:57.579255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:57.579255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:57.579255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:57.579255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:57.579255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:57.579255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:57.579255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:57.579255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:57.579255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:59:57.579255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:57.579255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 344/668] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 06:59:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:57.579255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff29ef2-bdb0-487a-a833-0fd0e86187c3 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:57.630848 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:57.692809 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba2117-a362-4c98-bb79-e625d928e048 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:57.693009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff29ef2-bdb0-487a-a833-0fd0e86187c3 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:57.729404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac81b9b-1d93-4deb-a533-df0fe650303c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:57.730093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac81b9b-1d93-4deb-a533-df0fe650303c None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:57.730372 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac81b9b-1d93-4deb-a533-df0fe650303c None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:57.730641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac81b9b-1d93-4deb-a533-df0fe650303c None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:57.731332 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac81b9b-1d93-4deb-a533-df0fe650303c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:57.731518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264ac1a-6fcd-447a-8e3b-b502a1118268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:57.731706 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac81b9b-1d93-4deb-a533-df0fe650303c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:57.732149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264ac1a-6fcd-447a-8e3b-b502a1118268 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:57.732430 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264ac1a-6fcd-447a-8e3b-b502a1118268 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:57.732705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264ac1a-6fcd-447a-8e3b-b502a1118268 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:57.733099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac81b9b-1d93-4deb-a533-df0fe650303c None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:57.740333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264ac1a-6fcd-447a-8e3b-b502a1118268 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:57.742271 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264ac1a-6fcd-447a-8e3b-b502a1118268 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:57.750616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 345/669] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 06:59:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:57.750616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac81b9b-1d93-4deb-a533-df0fe650303c None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:57.754146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-f264ac1a-6fcd-447a-8e3b-b502a1118268 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:57.754146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:57.754146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:57.754146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:57.754146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:57.754146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:57.754146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:57.754146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:57.754146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:57.754146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:57.754146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:57.754146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:57.754146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:59:57.754146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:59:57.754146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:59:57.755094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:59:57.755094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:57.755094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:57.755094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:57.755094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:57.755094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:57.755094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:57.755094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:57.755094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:57.755094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:57.755094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:57.755094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:57.755094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:57.755094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:57.755094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:57.755094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:57.755094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:57.756012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:59:57.756012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:59:57.756012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:59:57.756012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:59:57.756012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:59:57.756012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:59:57.756012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:57.756012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:57.756012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 325/670] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:57.756493 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264ac1a-6fcd-447a-8e3b-b502a1118268 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:57.795887 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:57.864188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac81b9b-1d93-4deb-a533-df0fe650303c None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:57.868436 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264ac1a-6fcd-447a-8e3b-b502a1118268 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:57.877632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:57.903865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec21a4d-88a2-4ced-a7e4-f8a138d99bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:57.904479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec21a4d-88a2-4ced-a7e4-f8a138d99bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:57.904760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec21a4d-88a2-4ced-a7e4-f8a138d99bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:57.905030 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec21a4d-88a2-4ced-a7e4-f8a138d99bc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:57.905382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47553426-a6db-42b1-90e1-41643f76ed93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:57.905990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47553426-a6db-42b1-90e1-41643f76ed93 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:57.906266 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47553426-a6db-42b1-90e1-41643f76ed93 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:57.906535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47553426-a6db-42b1-90e1-41643f76ed93 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:57.907192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47553426-a6db-42b1-90e1-41643f76ed93 None admin] RBAC: Adding query filter params (name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:57.907522 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47553426-a6db-42b1-90e1-41643f76ed93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:57.908919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47553426-a6db-42b1-90e1-41643f76ed93 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:57.917277 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 326/671] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 06:59:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:57.918077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47553426-a6db-42b1-90e1-41643f76ed93 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:57.984034 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec21a4d-88a2-4ced-a7e4-f8a138d99bc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:57.985580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec21a4d-88a2-4ced-a7e4-f8a138d99bc0 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:58.025043 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47553426-a6db-42b1-90e1-41643f76ed93 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:58.025292 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:58.061292 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf372fb8-2e92-4fbe-ac05-bdd8424ea585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:58.062017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf372fb8-2e92-4fbe-ac05-bdd8424ea585 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:58.062381 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf372fb8-2e92-4fbe-ac05-bdd8424ea585 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:58.062731 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf372fb8-2e92-4fbe-ac05-bdd8424ea585 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:58.063493 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf372fb8-2e92-4fbe-ac05-bdd8424ea585 None admin] RBAC: Adding query filter params (role.id=2d62b8b8395745c7983c6c8d100d985b, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=57a5f04cb9674b86a6b4b9a472e1afc0) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:58.063802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf372fb8-2e92-4fbe-ac05-bdd8424ea585 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:58.065248 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf372fb8-2e92-4fbe-ac05-bdd8424ea585 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:58.071405 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 327/672] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:59:57 2022] GET /identity/v3/role_assignments?user.id=57a5f04cb9674b86a6b4b9a472e1afc0&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2d62b8b8395745c7983c6c8d100d985b => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:58.072201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf372fb8-2e92-4fbe-ac05-bdd8424ea585 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:58.106695 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 346/673] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:58.108032 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec21a4d-88a2-4ced-a7e4-f8a138d99bc0 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:58.127258 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:58.219465 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec21a4d-88a2-4ced-a7e4-f8a138d99bc0 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:58.221328 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf372fb8-2e92-4fbe-ac05-bdd8424ea585 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:58.265179 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32cf01-c471-4144-9cd6-ebae7c3f28a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:58.265822 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32cf01-c471-4144-9cd6-ebae7c3f28a3 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:58.266103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32cf01-c471-4144-9cd6-ebae7c3f28a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:58.266374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32cf01-c471-4144-9cd6-ebae7c3f28a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:58.266580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d633ea-dd78-4320-979c-28b8a63390e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:58.267080 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32cf01-c471-4144-9cd6-ebae7c3f28a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:58.267381 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32cf01-c471-4144-9cd6-ebae7c3f28a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:58.267911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d633ea-dd78-4320-979c-28b8a63390e2 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:58.269106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d633ea-dd78-4320-979c-28b8a63390e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:58.269219 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32cf01-c471-4144-9cd6-ebae7c3f28a3 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:58.269696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d633ea-dd78-4320-979c-28b8a63390e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:58.273850 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:58.281624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 328/674] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 06:59:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:58.282460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32cf01-c471-4144-9cd6-ebae7c3f28a3 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:58.284526 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d633ea-dd78-4320-979c-28b8a63390e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:58.286075 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d633ea-dd78-4320-979c-28b8a63390e2 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:58.303795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-02d633ea-dd78-4320-979c-28b8a63390e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:58.303795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:58.303795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:58.303795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:58.303795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:58.303795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:58.303795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:58.303795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:58.303795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:58.303795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:58.303795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:58.303795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:58.303795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:58.303795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:58.303795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:58.303795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:58.304787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:58.304787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:58.304787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:58.304787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:58.304787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:58.304787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:58.304787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:58.304787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:58.304787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:58.304787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:58.304787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:58.304787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:58.304787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:58.304787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:58.304787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:58.304787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:58.304787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:58.305684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:58.305684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:58.305684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:58.305684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 347/675] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:58.306793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cc2fa-40c2-4454-b5b9-10e9d3b48fb0 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:58.306793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cc2fa-40c2-4454-b5b9-10e9d3b48fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:58.307237 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cc2fa-40c2-4454-b5b9-10e9d3b48fb0 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:58.308607 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 348/676] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 06:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:59:58.309301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cc2fa-40c2-4454-b5b9-10e9d3b48fb0 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:58.392898 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:58.401609 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32cf01-c471-4144-9cd6-ebae7c3f28a3 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:58.423624 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cc2fa-40c2-4454-b5b9-10e9d3b48fb0 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:58.438363 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d62442-6bae-4a02-b45e-35462b9d3548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:58.439047 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d62442-6bae-4a02-b45e-35462b9d3548 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:58.439326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d62442-6bae-4a02-b45e-35462b9d3548 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:58.439627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d62442-6bae-4a02-b45e-35462b9d3548 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:58.452305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d62442-6bae-4a02-b45e-35462b9d3548 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:58.454742 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d62442-6bae-4a02-b45e-35462b9d3548 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:58.458516 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5693c2-7098-41f9-a044-c21b19f489d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:58.459125 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5693c2-7098-41f9-a044-c21b19f489d7 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:58.459424 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5693c2-7098-41f9-a044-c21b19f489d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:58.459733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5693c2-7098-41f9-a044-c21b19f489d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:58.460461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5693c2-7098-41f9-a044-c21b19f489d7 None admin] RBAC: Adding query filter params (role.id=2d62b8b8395745c7983c6c8d100d985b, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=a26f09e892a74d9eab97e73cce9d062a) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:58.460770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5693c2-7098-41f9-a044-c21b19f489d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:58.462223 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5693c2-7098-41f9-a044-c21b19f489d7 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:58.468310 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 349/677] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:59:58 2022] GET /identity/v3/role_assignments?user.id=a26f09e892a74d9eab97e73cce9d062a&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2d62b8b8395745c7983c6c8d100d985b => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:58.469108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5693c2-7098-41f9-a044-c21b19f489d7 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:58.471700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-f3d62442-6bae-4a02-b45e-35462b9d3548 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:58.471700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:58.471700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:58.471700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:58.471700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:58.471700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:58.471700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:58.471700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:58.471700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:58.471700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:58.471700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:58.471700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:58.471700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:58.471700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:58.471700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:58.471700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:58.472913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:58.472913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:58.472913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:58.472913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:58.472913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:58.472913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:58.472913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:58.472913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:58.472913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:58.472913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:58.472913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:58.472913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:58.472913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:58.472913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:58.472913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:58.472913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:58.472913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:58.473856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:58.473856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:58.473856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:58.474038 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 329/678] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:58.475311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf7afb-9d27-401c-a41c-5c9d05cc6688 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:58.475629 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf7afb-9d27-401c-a41c-5c9d05cc6688 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:58.475970 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf7afb-9d27-401c-a41c-5c9d05cc6688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:58.500621 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-8bcf7afb-9d27-401c-a41c-5c9d05cc6688 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:59:58.505348 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8bcf7afb-9d27-401c-a41c-5c9d05cc6688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:59:58.546182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:58.603442 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5693c2-7098-41f9-a044-c21b19f489d7 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:58.611437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 330/679] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 06:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:59:58.611437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf7afb-9d27-401c-a41c-5c9d05cc6688 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:58.649537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ff653-a759-414a-983e-e9dbaa3c0927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:58.650634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ff653-a759-414a-983e-e9dbaa3c0927 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:58.651104 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ff653-a759-414a-983e-e9dbaa3c0927 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:58.651672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ff653-a759-414a-983e-e9dbaa3c0927 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:58.652583 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ff653-a759-414a-983e-e9dbaa3c0927 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:58.653098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ff653-a759-414a-983e-e9dbaa3c0927 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:58.654680 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ff653-a759-414a-983e-e9dbaa3c0927 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:58.669766 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 350/680] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:58.675291 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ff653-a759-414a-983e-e9dbaa3c0927 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:58.725457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:58.752765 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf7afb-9d27-401c-a41c-5c9d05cc6688 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:58.792875 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72975405-4456-47cd-9b5f-5dfd4eed6a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:58.793951 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72975405-4456-47cd-9b5f-5dfd4eed6a08 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:58.794413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72975405-4456-47cd-9b5f-5dfd4eed6a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:58.794864 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72975405-4456-47cd-9b5f-5dfd4eed6a08 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:58.795803 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72975405-4456-47cd-9b5f-5dfd4eed6a08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:58.796331 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72975405-4456-47cd-9b5f-5dfd4eed6a08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:58.797979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72975405-4456-47cd-9b5f-5dfd4eed6a08 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:58.803318 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ff653-a759-414a-983e-e9dbaa3c0927 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:58.812744 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 331/681] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:58.820039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:58.821585 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72975405-4456-47cd-9b5f-5dfd4eed6a08 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:58.849372 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6dcbe5-8f4c-4151-a073-dd05d412faca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:58.850199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6dcbe5-8f4c-4151-a073-dd05d412faca None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:58.850575 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6dcbe5-8f4c-4151-a073-dd05d412faca None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:58.850963 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6dcbe5-8f4c-4151-a073-dd05d412faca None admin] PATH_INFO: `/v3/users/swift` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:58.860435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6dcbe5-8f4c-4151-a073-dd05d412faca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:58.862705 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6dcbe5-8f4c-4151-a073-dd05d412faca None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:58.875710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-5a6dcbe5-8f4c-4151-a073-dd05d412faca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 06:59:58.875710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:58.875710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:58.875710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:58.875710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:58.875710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:58.875710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:58.875710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:58.875710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:58.875710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:58.875710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:58.875710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:58.875710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:59:58.875710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:59:58.875710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:59:58.875710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:59:58.876934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:58.876934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:58.876934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:59:58.876934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:58.876934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:59:58.876934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:59:58.876934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:58.876934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:58.876934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:58.876934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:58.876934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:58.876934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:58.876934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:58.876934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:58.876934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:58.876934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:58.876934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:58.876934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:58.878086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:58.878086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:58.878086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:59:58.878086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:59:58.878086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:59:58.878086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:59:58.878086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:59:58.878086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:59:58.878086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 06:59:58.878086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 06:59:58.880477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 351/682] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 06:59:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:58.886182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6dcbe5-8f4c-4151-a073-dd05d412faca None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:58.946322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72975405-4456-47cd-9b5f-5dfd4eed6a08 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:58.989673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98fa61-383e-4b90-bbe7-507f848d6f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:58.990310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98fa61-383e-4b90-bbe7-507f848d6f0e None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:58.990585 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98fa61-383e-4b90-bbe7-507f848d6f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:58.990854 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98fa61-383e-4b90-bbe7-507f848d6f0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:58.996643 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:59.001424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98fa61-383e-4b90-bbe7-507f848d6f0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:59.003006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98fa61-383e-4b90-bbe7-507f848d6f0e None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:59.017498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-5c98fa61-383e-4b90-bbe7-507f848d6f0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:59.017498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:59.017498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:59.017498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:59.017498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:59.017498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:59.017498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:59.017498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:59.017498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:59.017498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:59.017498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:59.017498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:59.017498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:59:59.017498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:59:59.017498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:59:59.019151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:59:59.019151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:59.019151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:59.019151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:59.019151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:59.019151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:59.019151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:59.019151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:59.019151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:59.019151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:59.019151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:59.019151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:59.019151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:59.019151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:59.019151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:59.019151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:59.019151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:59.020838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:59:59.020838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:59:59.020838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:59:59.020838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:59:59.020838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:59:59.020838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:59:59.020838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:59:59.020838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:59.020838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 332/683] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 06:59:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:59.023944 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98fa61-383e-4b90-bbe7-507f848d6f0e None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:59.033644 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6dcbe5-8f4c-4151-a073-dd05d412faca None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:59.079628 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbeffc-07c9-477f-af8a-81e54f58e021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:59.081217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbeffc-07c9-477f-af8a-81e54f58e021 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:59.081863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbeffc-07c9-477f-af8a-81e54f58e021 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:59.082469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbeffc-07c9-477f-af8a-81e54f58e021 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:59.083775 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbeffc-07c9-477f-af8a-81e54f58e021 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:59.084423 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbeffc-07c9-477f-af8a-81e54f58e021 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:59.086099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbeffc-07c9-477f-af8a-81e54f58e021 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:59.108282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 352/684] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Aug 23 06:59:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:59.109247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbeffc-07c9-477f-af8a-81e54f58e021 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:59.152826 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:59.155471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98fa61-383e-4b90-bbe7-507f848d6f0e None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:59.206625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4901a1e-c872-4f81-abaf-2cbe28a7e4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:59.208159 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4901a1e-c872-4f81-abaf-2cbe28a7e4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:59.208426 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4901a1e-c872-4f81-abaf-2cbe28a7e4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:59.208672 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4901a1e-c872-4f81-abaf-2cbe28a7e4e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:59.227329 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4901a1e-c872-4f81-abaf-2cbe28a7e4e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:59:59.240486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-d4901a1e-c872-4f81-abaf-2cbe28a7e4e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:59:59.240486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:59.240486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:59.240486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:59.240486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:59.240486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:59.240486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:59.240486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:59.240486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:59.240486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:59.240486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:59.240486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:59.240486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:59:59.240486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:59:59.240486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:59:59.240486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:59:59.241504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:59:59.241504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:59:59.241504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:59:59.241504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:59:59.241504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:59.241504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:59.241504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:59.241504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:59.241504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:59.241504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:59.241504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:59.241504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:59.241504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:59.241504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:59.241504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:59.241504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:59.242336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:59.242336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:59.242336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:59.242336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:59.242336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:59:59.242336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:59:59.242336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:59:59.242336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:59:59.242336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:59:59.242336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:59:59.242336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:59:59.242336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:59.245392 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 333/685] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 06:59:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:59.246612 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4901a1e-c872-4f81-abaf-2cbe28a7e4e7 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:59.254017 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbeffc-07c9-477f-af8a-81e54f58e021 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:59.303406 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373b671-c112-4b73-bcc3-6965a54a0132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:59.304352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373b671-c112-4b73-bcc3-6965a54a0132 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:59.304668 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373b671-c112-4b73-bcc3-6965a54a0132 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:59.305090 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373b671-c112-4b73-bcc3-6965a54a0132 None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:59.307567 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4373b671-c112-4b73-bcc3-6965a54a0132 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:59.307567 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4373b671-c112-4b73-bcc3-6965a54a0132 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:59.307904 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4373b671-c112-4b73-bcc3-6965a54a0132 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:59.322317 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 353/686] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 06:59:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:59.323282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373b671-c112-4b73-bcc3-6965a54a0132 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:59.339562 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:59.413505 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4901a1e-c872-4f81-abaf-2cbe28a7e4e7 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:59.450801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:59.465051 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ce5a1-9c47-4d50-8c4e-89f1ff771e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:59.465981 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ce5a1-9c47-4d50-8c4e-89f1ff771e9f None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:59.466101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373b671-c112-4b73-bcc3-6965a54a0132 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:59.466565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ce5a1-9c47-4d50-8c4e-89f1ff771e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:59.467021 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ce5a1-9c47-4d50-8c4e-89f1ff771e9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:59.508989 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ae1cc-9884-4880-8e62-60785c3e9959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:59.510957 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ae1cc-9884-4880-8e62-60785c3e9959 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:59.511213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ae1cc-9884-4880-8e62-60785c3e9959 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:59.511609 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ae1cc-9884-4880-8e62-60785c3e9959 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:59.512349 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ae1cc-9884-4880-8e62-60785c3e9959 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:59.512721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ae1cc-9884-4880-8e62-60785c3e9959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:59.514242 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ae1cc-9884-4880-8e62-60785c3e9959 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:59.524438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 354/687] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 06:59:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:59.525391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ae1cc-9884-4880-8e62-60785c3e9959 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:59.560332 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ce5a1-9c47-4d50-8c4e-89f1ff771e9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:59.562779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ce5a1-9c47-4d50-8c4e-89f1ff771e9f None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:59.631107 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:59.652311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ae1cc-9884-4880-8e62-60785c3e9959 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:59.689227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 334/688] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 06:59:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:59:59.690914 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ce5a1-9c47-4d50-8c4e-89f1ff771e9f None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:59.695917 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e105bde-6dca-4bfa-83c9-532a242dec8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:59.696887 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e105bde-6dca-4bfa-83c9-532a242dec8c None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:59.697378 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e105bde-6dca-4bfa-83c9-532a242dec8c None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:59.697862 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e105bde-6dca-4bfa-83c9-532a242dec8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:59.698805 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e105bde-6dca-4bfa-83c9-532a242dec8c None admin] RBAC: Adding query filter params (role.id=2d62b8b8395745c7983c6c8d100d985b, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=938364c02d8d4dd2ae832c5129be3cbe) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:59:59.699315 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e105bde-6dca-4bfa-83c9-532a242dec8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:59.701133 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e105bde-6dca-4bfa-83c9-532a242dec8c None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:59.711028 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 355/689] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 06:59:59 2022] GET /identity/v3/role_assignments?user.id=938364c02d8d4dd2ae832c5129be3cbe&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2d62b8b8395745c7983c6c8d100d985b => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:59:59.712583 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e105bde-6dca-4bfa-83c9-532a242dec8c None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:59:59.736203 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:59:59.856890 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ce5a1-9c47-4d50-8c4e-89f1ff771e9f None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:59.873055 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e105bde-6dca-4bfa-83c9-532a242dec8c None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:59:59.924939 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0308e5-6cb2-4955-a237-540803ef5b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:59.925972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0308e5-6cb2-4955-a237-540803ef5b95 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:59.926705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0308e5-6cb2-4955-a237-540803ef5b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:59.927161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0308e5-6cb2-4955-a237-540803ef5b95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:59.950271 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0308e5-6cb2-4955-a237-540803ef5b95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:59.960970 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87963dbf-37e0-489e-9dea-eb55ec15f0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:59:59.961691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87963dbf-37e0-489e-9dea-eb55ec15f0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:59:59.961978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87963dbf-37e0-489e-9dea-eb55ec15f0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:59:59.962255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87963dbf-37e0-489e-9dea-eb55ec15f0b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:59:59.963644 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0308e5-6cb2-4955-a237-540803ef5b95 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:59.982517 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87963dbf-37e0-489e-9dea-eb55ec15f0b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:59:59.984462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87963dbf-37e0-489e-9dea-eb55ec15f0b6 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:59:59.987015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-3a0308e5-6cb2-4955-a237-540803ef5b95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:59.987015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:59:59.987015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:59:59.987015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:59:59.987015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:59:59.987015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:59:59.987015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:59:59.987015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:59:59.987015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:59:59.987015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:59:59.987015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:59:59.987015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:59:59.987015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:59:59.987015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:59:59.987015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:59:59.987015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:59:59.988345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:59:59.988345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:59:59.988345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:59:59.988345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:59:59.988345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:59:59.988345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:59:59.988345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:59:59.988345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 06:59:59.988345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:59:59.988345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:59:59.988345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:59:59.988345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:59:59.988345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:59:59.988345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:59:59.988345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:59:59.988345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:59:59.988345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:59:59.989681 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:59:59.989681 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:59:59.989681 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 06:59:59.989681 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 335/690] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:59:59.994947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0308e5-6cb2-4955-a237-540803ef5b95 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:00.007654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-87963dbf-37e0-489e-9dea-eb55ec15f0b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:00.007654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:00.007654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:00.007654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:00.007654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:00.007654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:00.007654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:00.007654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:00.007654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:00.007654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:00.007654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:00.007654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:00.007654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:00.007654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:00.007654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:00.007654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:00.008724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:00.008724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:00.008724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:00.008724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:00.008724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:00.008724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:00.008724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:00.008724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:00.008724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:00.008724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:00.008724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:00.008724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:00.008724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:00.008724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:00.008724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:00.008724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:00.008724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:00.009702 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:00.009702 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:00.009702 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:00.011120 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:00.012213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 356/691] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 06:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:00.018524 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87963dbf-37e0-489e-9dea-eb55ec15f0b6 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:00.092260 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:00.117570 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0308e5-6cb2-4955-a237-540803ef5b95 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:00.131335 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87963dbf-37e0-489e-9dea-eb55ec15f0b6 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:00.154751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439f40f-f6fd-4f09-aaf3-5a229e828700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:00.155695 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439f40f-f6fd-4f09-aaf3-5a229e828700 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:00.156220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439f40f-f6fd-4f09-aaf3-5a229e828700 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:00.156681 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439f40f-f6fd-4f09-aaf3-5a229e828700 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:00.157605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439f40f-f6fd-4f09-aaf3-5a229e828700 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:00.158090 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439f40f-f6fd-4f09-aaf3-5a229e828700 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:00.159843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439f40f-f6fd-4f09-aaf3-5a229e828700 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:00.168667 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5921d-b2f0-4a98-9d5e-9e51bb98bc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:00.169289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5921d-b2f0-4a98-9d5e-9e51bb98bc54 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:00.169568 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5921d-b2f0-4a98-9d5e-9e51bb98bc54 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:00.169840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5921d-b2f0-4a98-9d5e-9e51bb98bc54 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:00.170499 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce5921d-b2f0-4a98-9d5e-9e51bb98bc54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:00.170801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce5921d-b2f0-4a98-9d5e-9e51bb98bc54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:00.172205 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce5921d-b2f0-4a98-9d5e-9e51bb98bc54 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:00.175618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 336/692] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 06:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:00.181231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439f40f-f6fd-4f09-aaf3-5a229e828700 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:00.186576 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 357/693] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:00.191893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5921d-b2f0-4a98-9d5e-9e51bb98bc54 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:00.259900 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:00.292642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439f40f-f6fd-4f09-aaf3-5a229e828700 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:00.297361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5921d-b2f0-4a98-9d5e-9e51bb98bc54 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:00.326020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:00.336555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21c2b3-5bd0-4beb-8cad-119b469e462e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:00.337197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21c2b3-5bd0-4beb-8cad-119b469e462e None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:00.337474 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21c2b3-5bd0-4beb-8cad-119b469e462e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:00.337744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21c2b3-5bd0-4beb-8cad-119b469e462e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:00.338047 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf369367-6407-4512-b67f-add7fee4cb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:00.338921 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf369367-6407-4512-b67f-add7fee4cb58 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:00.339405 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf369367-6407-4512-b67f-add7fee4cb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:00.339950 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf369367-6407-4512-b67f-add7fee4cb58 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:00.345068 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21c2b3-5bd0-4beb-8cad-119b469e462e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:00.346604 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21c2b3-5bd0-4beb-8cad-119b469e462e None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:00.348740 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf369367-6407-4512-b67f-add7fee4cb58 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:00.350553 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf369367-6407-4512-b67f-add7fee4cb58 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:00.358774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-7c21c2b3-5bd0-4beb-8cad-119b469e462e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:00.358774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:00.358774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:00.358774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:00.358774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:00.358774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:00.358774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:00.358774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:00.358774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:00.358774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:00.358774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:00.358774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:00.358774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:00:00.358774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:00:00.358774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:00:00.359750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:00:00.359750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:00.359750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:00.359750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:00.359750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:00.359750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:00.359750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:00.359750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:00.359750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:00.359750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:00.359750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:00.359750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:00.359750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:00.359750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:00.359750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:00.359750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:00.359750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:00.361021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:00:00.361021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:00:00.361021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:00:00.361021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:00:00.361021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:00:00.361021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:00:00.361021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:00.361021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:00.361021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 358/694] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:00.361513 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bef3f0-3f98-4760-bfdd-15f0d42c7e40 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:00.361808 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bef3f0-3f98-4760-bfdd-15f0d42c7e40 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:00.361976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bef3f0-3f98-4760-bfdd-15f0d42c7e40 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:00.366725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-bf369367-6407-4512-b67f-add7fee4cb58 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:00:00.366725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:00.366725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:00.366725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:00.366725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:00.366725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:00.366725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:00.366725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:00.366725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:00.366725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:00.366725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:00.366725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:00.366725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:00:00.366725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:00:00.366725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:00:00.366725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:00:00.368036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:00.368036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:00.368036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:00:00.368036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:00.368036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:00:00.368036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:00.368036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:00.368036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:00.368036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:00.368036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:00.368036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:00.368036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:00.368036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:00.368036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:00.368036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:00.368036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:00.368036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:00.368036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:00.369383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:00.369383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:00.369383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:00:00.369383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:00:00.369383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:00:00.369383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:00:00.369383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:00:00.369383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:00:00.369383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:00:00.369383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:00.369925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 359/695] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:00.369925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bef3f0-3f98-4760-bfdd-15f0d42c7e40 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:00.371746 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 337/696] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Aug 23 07:00:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:00.373750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499dbfb-23ce-4cee-9746-891c9315d61e None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:00.374250 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499dbfb-23ce-4cee-9746-891c9315d61e None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:00.374721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499dbfb-23ce-4cee-9746-891c9315d61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:00.394057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-9499dbfb-23ce-4cee-9746-891c9315d61e None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:00.395788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9499dbfb-23ce-4cee-9746-891c9315d61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:00.468552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 338/697] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:00:00.469866 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499dbfb-23ce-4cee-9746-891c9315d61e None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:00.479363 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bef3f0-3f98-4760-bfdd-15f0d42c7e40 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:00.514930 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39120a8e-d2cf-4b4f-86b9-91bb410d3c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:00.515485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39120a8e-d2cf-4b4f-86b9-91bb410d3c58 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:00.515765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39120a8e-d2cf-4b4f-86b9-91bb410d3c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:00.516034 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39120a8e-d2cf-4b4f-86b9-91bb410d3c58 None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:00.516923 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39120a8e-d2cf-4b4f-86b9-91bb410d3c58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:00.517141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39120a8e-d2cf-4b4f-86b9-91bb410d3c58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:00.518572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39120a8e-d2cf-4b4f-86b9-91bb410d3c58 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:00.535655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 360/698] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:00:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:00.537125 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:00.538233 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39120a8e-d2cf-4b4f-86b9-91bb410d3c58 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:00.578768 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499dbfb-23ce-4cee-9746-891c9315d61e None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:00.603567 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:00.624793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ac3b5-9699-42df-b17d-299c11cda6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:00.625414 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ac3b5-9699-42df-b17d-299c11cda6df None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:00.625687 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ac3b5-9699-42df-b17d-299c11cda6df None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:00.625958 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ac3b5-9699-42df-b17d-299c11cda6df None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:00.626689 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ac3b5-9699-42df-b17d-299c11cda6df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:00.626992 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ac3b5-9699-42df-b17d-299c11cda6df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:00.633509 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ac3b5-9699-42df-b17d-299c11cda6df None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:00.660146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 339/699] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Aug 23 07:00:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:00.668446 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ac3b5-9699-42df-b17d-299c11cda6df None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:00.689781 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39120a8e-d2cf-4b4f-86b9-91bb410d3c58 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:00.727325 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bd9d3-9c3d-4175-bf2b-ba0969fa60f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:00.728275 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bd9d3-9c3d-4175-bf2b-ba0969fa60f4 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:00.728554 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bd9d3-9c3d-4175-bf2b-ba0969fa60f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:00.728831 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bd9d3-9c3d-4175-bf2b-ba0969fa60f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:00.729563 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443bd9d3-9c3d-4175-bf2b-ba0969fa60f4 None admin] RBAC: Adding query filter params (role.id=2d62b8b8395745c7983c6c8d100d985b, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=fd88ec8b477443af95bff9af03471adf) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:00.729870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443bd9d3-9c3d-4175-bf2b-ba0969fa60f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:00.731429 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443bd9d3-9c3d-4175-bf2b-ba0969fa60f4 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:00.738238 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 361/700] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:00:00 2022] GET /identity/v3/role_assignments?user.id=fd88ec8b477443af95bff9af03471adf&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2d62b8b8395745c7983c6c8d100d985b => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:00.813606 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bd9d3-9c3d-4175-bf2b-ba0969fa60f4 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:00.911638 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ac3b5-9699-42df-b17d-299c11cda6df None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:00.935216 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:00.965328 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56971282-a873-4d29-99eb-da6bc084f8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:00.965968 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56971282-a873-4d29-99eb-da6bc084f8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:00.966251 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56971282-a873-4d29-99eb-da6bc084f8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:00.966558 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56971282-a873-4d29-99eb-da6bc084f8e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:00.990316 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bd9d3-9c3d-4175-bf2b-ba0969fa60f4 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:01.004989 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:01.033367 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce5239-0fd5-4f03-9b58-b075362dc6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:01.034101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce5239-0fd5-4f03-9b58-b075362dc6f2 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:01.034383 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce5239-0fd5-4f03-9b58-b075362dc6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:01.034657 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce5239-0fd5-4f03-9b58-b075362dc6f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:01.045919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56971282-a873-4d29-99eb-da6bc084f8e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:01.046565 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce5239-0fd5-4f03-9b58-b075362dc6f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:01.047391 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56971282-a873-4d29-99eb-da6bc084f8e3 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:01.048037 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce5239-0fd5-4f03-9b58-b075362dc6f2 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:01.063244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-23ce5239-0fd5-4f03-9b58-b075362dc6f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:01.063244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:01.063244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:01.063244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:01.063244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:01.063244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:01.063244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:01.063244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:01.063244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:01.063244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:01.063244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:01.063244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:01.063244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:01.063244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:01.063244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:01.063244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:01.064327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:01.064327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:01.064327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:01.064327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:01.064327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:01.064327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:01.064327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:01.064327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:01.064327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:01.064327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:01.064327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:01.064327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:01.064327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:01.064327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:01.064327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:01.064327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:01.064327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:01.065323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:01.065323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:01.065323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:01.065323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 362/701] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:01.071660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac113046-ae5e-49be-88d4-0641b7520f2e None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:01.071660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac113046-ae5e-49be-88d4-0641b7520f2e None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:01.071660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac113046-ae5e-49be-88d4-0641b7520f2e None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:01.071660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 363/702] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:01.072123 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac113046-ae5e-49be-88d4-0641b7520f2e None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:01.178450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 340/703] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:00:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:00:01.184214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66676079-548b-4101-bf4f-17c70686db6d None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:01.184514 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66676079-548b-4101-bf4f-17c70686db6d None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:01.184798 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66676079-548b-4101-bf4f-17c70686db6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:01.203265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac113046-ae5e-49be-88d4-0641b7520f2e None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:01.204029 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-66676079-548b-4101-bf4f-17c70686db6d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:01.205345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-66676079-548b-4101-bf4f-17c70686db6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:01.217336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:01.243202 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372a427-dd8c-43dc-94ea-98d53ec66c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:01.244315 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372a427-dd8c-43dc-94ea-98d53ec66c79 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:01.244887 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372a427-dd8c-43dc-94ea-98d53ec66c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:01.245382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372a427-dd8c-43dc-94ea-98d53ec66c79 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:01.246234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372a427-dd8c-43dc-94ea-98d53ec66c79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:01.246737 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372a427-dd8c-43dc-94ea-98d53ec66c79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:01.248349 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372a427-dd8c-43dc-94ea-98d53ec66c79 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:01.262486 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 364/704] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:01.268398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372a427-dd8c-43dc-94ea-98d53ec66c79 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:01.282103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 341/705] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:00:01.283360 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66676079-548b-4101-bf4f-17c70686db6d None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:01.297891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:01.381621 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372a427-dd8c-43dc-94ea-98d53ec66c79 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:01.402189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66676079-548b-4101-bf4f-17c70686db6d None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:01.419266 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de8d65-16db-4a57-925e-edf45e486e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:01.420345 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de8d65-16db-4a57-925e-edf45e486e77 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:01.420825 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de8d65-16db-4a57-925e-edf45e486e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:01.421345 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de8d65-16db-4a57-925e-edf45e486e77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:01.432779 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de8d65-16db-4a57-925e-edf45e486e77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:01.434769 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de8d65-16db-4a57-925e-edf45e486e77 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:01.439264 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8751ac-d285-4e97-80de-b175ab514b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:01.439957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8751ac-d285-4e97-80de-b175ab514b5a None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:01.440237 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8751ac-d285-4e97-80de-b175ab514b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:01.440562 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8751ac-d285-4e97-80de-b175ab514b5a None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:01.441539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8751ac-d285-4e97-80de-b175ab514b5a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:01.443494 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8751ac-d285-4e97-80de-b175ab514b5a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:01.453433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-23de8d65-16db-4a57-925e-edf45e486e77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:01.453433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:01.453433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:01.453433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:01.453433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:01.453433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:01.453433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:01.453433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:01.453433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:01.453433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:01.453433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:01.453433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:01.453433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:01.453433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:01.453433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:01.453433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:01.454402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:01.454402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:01.454402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:01.454402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:01.454402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:01.454402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:01.454402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:01.454402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:01.454402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:01.454402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:01.454402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:01.454402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:01.454402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:01.454402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:01.454402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:01.454402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:01.454402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:01.455313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:01.455313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:01.455313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:01.457533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 365/706] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:01.464943 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de8d65-16db-4a57-925e-edf45e486e77 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:01.477879 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 342/707] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 23 07:00:01 2022] POST /identity/v3/users => generated 254 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:00:01.613863 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8751ac-d285-4e97-80de-b175ab514b5a None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:01.614884 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:01.617047 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:01.664430 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de8d65-16db-4a57-925e-edf45e486e77 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:01.706764 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc751f-5c20-45fb-9333-b08815bb52a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:01.707721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc751f-5c20-45fb-9333-b08815bb52a5 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:01.708006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc751f-5c20-45fb-9333-b08815bb52a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:01.708281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc751f-5c20-45fb-9333-b08815bb52a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:01.708941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc751f-5c20-45fb-9333-b08815bb52a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:01.709339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc751f-5c20-45fb-9333-b08815bb52a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:01.710857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc751f-5c20-45fb-9333-b08815bb52a5 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:01.724416 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 366/708] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:01.730398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc751f-5c20-45fb-9333-b08815bb52a5 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:01.752994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8751ac-d285-4e97-80de-b175ab514b5a None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:01.789561 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92bba76-25b7-4c4d-a1e7-1b20e4ae34a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:01.790548 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bba76-25b7-4c4d-a1e7-1b20e4ae34a9 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:01.791000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bba76-25b7-4c4d-a1e7-1b20e4ae34a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:01.791439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bba76-25b7-4c4d-a1e7-1b20e4ae34a9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:01.792177 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bba76-25b7-4c4d-a1e7-1b20e4ae34a9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:01.793821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bba76-25b7-4c4d-a1e7-1b20e4ae34a9 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:01.806300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-d92bba76-25b7-4c4d-a1e7-1b20e4ae34a9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:00:01.806300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:01.806300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:01.806300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:01.806300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:01.806300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:01.806300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:01.806300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:01.806300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:00:01.806300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:01.806300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:01.806300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:01.806300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:01.806300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:01.806300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:01.806300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:01.806300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:01.807385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:01.807385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:01.807385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:01.807385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:01.807385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:01.807385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:01.807385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:01.807385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:01.807385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:01.807385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:01.807385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:01.807385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:01.807385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:01.807385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:01.807385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:01.807385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:01.808283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:01.808283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:01.808283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:01.808283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:01.808283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:01.808283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:01.808283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:01.808283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:01.808283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:01.808283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:01.808283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:01.808283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:01.808283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:00:01.808283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:01.810895 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 343/709] 10.222.0.17 () {56 vars in 1129 bytes} [Tue Aug 23 07:00:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:01.817368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92bba76-25b7-4c4d-a1e7-1b20e4ae34a9 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:01.853149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc751f-5c20-45fb-9333-b08815bb52a5 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:01.898743 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad877c3c-7166-46f5-8796-20a52da3a8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:01.899999 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad877c3c-7166-46f5-8796-20a52da3a8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:01.900478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad877c3c-7166-46f5-8796-20a52da3a8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:01.901225 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad877c3c-7166-46f5-8796-20a52da3a8e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:01.906031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:01.911603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:01.912095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad877c3c-7166-46f5-8796-20a52da3a8e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:01.914377 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad877c3c-7166-46f5-8796-20a52da3a8e0 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:01.929033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-ad877c3c-7166-46f5-8796-20a52da3a8e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:01.929033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:01.929033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:01.929033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:01.929033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:01.929033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:01.929033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:01.929033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:01.929033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:01.929033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:01.929033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:01.929033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:01.929033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:00:01.929033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:00:01.929033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:00:01.930236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:00:01.930236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:01.930236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:01.930236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:01.930236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:01.930236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:01.930236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:01.930236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:01.930236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:01.930236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:01.930236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:01.930236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:01.930236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:01.930236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:01.930236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:01.930236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:01.930236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:01.931353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:00:01.931353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:00:01.931353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:00:01.931353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:00:01.931353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:00:01.931353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:00:01.931353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:01.931353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:01.934134 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 367/710] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:01.937863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad877c3c-7166-46f5-8796-20a52da3a8e0 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:01.968743 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92bba76-25b7-4c4d-a1e7-1b20e4ae34a9 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:02.022320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8562371-4edd-46ce-b078-dceb07c33563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:02.023455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8562371-4edd-46ce-b078-dceb07c33563 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:02.023975 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8562371-4edd-46ce-b078-dceb07c33563 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:02.024441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8562371-4edd-46ce-b078-dceb07c33563 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:02.025682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8562371-4edd-46ce-b078-dceb07c33563 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:02.027097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8562371-4edd-46ce-b078-dceb07c33563 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:02.029090 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8562371-4edd-46ce-b078-dceb07c33563 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:02.042195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 344/711] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Aug 23 07:00:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:02.043846 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8562371-4edd-46ce-b078-dceb07c33563 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:02.125229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad877c3c-7166-46f5-8796-20a52da3a8e0 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:02.161571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8562371-4edd-46ce-b078-dceb07c33563 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:02.190013 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e75290-5923-4b05-971f-f4b2ccf68771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:02.190627 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e75290-5923-4b05-971f-f4b2ccf68771 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:02.190910 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e75290-5923-4b05-971f-f4b2ccf68771 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:02.191183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e75290-5923-4b05-971f-f4b2ccf68771 None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:02.191969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e75290-5923-4b05-971f-f4b2ccf68771 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:02.192288 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e75290-5923-4b05-971f-f4b2ccf68771 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:02.193763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e75290-5923-4b05-971f-f4b2ccf68771 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:02.195523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:02.201179 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef4b23-8fcc-44d1-a42b-172f82a5fc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:02.202021 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef4b23-8fcc-44d1-a42b-172f82a5fc37 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:02.203677 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef4b23-8fcc-44d1-a42b-172f82a5fc37 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:02.203677 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef4b23-8fcc-44d1-a42b-172f82a5fc37 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:02.204043 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef4b23-8fcc-44d1-a42b-172f82a5fc37 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:02.204508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef4b23-8fcc-44d1-a42b-172f82a5fc37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:02.206189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef4b23-8fcc-44d1-a42b-172f82a5fc37 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:02.207094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 368/712] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:00:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:02.208598 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589220e-4dab-480b-880e-bef3e841a280 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:02.208889 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589220e-4dab-480b-880e-bef3e841a280 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:02.209163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589220e-4dab-480b-880e-bef3e841a280 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:02.210693 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 369/713] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:02.219550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589220e-4dab-480b-880e-bef3e841a280 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:02.219800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 345/714] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Aug 23 07:00:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:02.220147 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5d135-4d24-4aa2-8e50-aa6efcd8d439 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:02.220420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5d135-4d24-4aa2-8e50-aa6efcd8d439 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:02.222932 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5d135-4d24-4aa2-8e50-aa6efcd8d439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:02.258530 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-4fa5d135-4d24-4aa2-8e50-aa6efcd8d439 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:02.259773 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-4fa5d135-4d24-4aa2-8e50-aa6efcd8d439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:02.328514 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:02.378719 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589220e-4dab-480b-880e-bef3e841a280 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:02.401578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 346/715] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:00:02.433167 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d477c1-dd14-42f1-9e0e-1872b49d9883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:02.433689 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d477c1-dd14-42f1-9e0e-1872b49d9883 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:02.433962 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d477c1-dd14-42f1-9e0e-1872b49d9883 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:02.434231 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d477c1-dd14-42f1-9e0e-1872b49d9883 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:02.434963 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d477c1-dd14-42f1-9e0e-1872b49d9883 None admin] RBAC: Adding query filter params (role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=a8d2cd93042c4583b63afe629d47fce0) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:02.435266 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d477c1-dd14-42f1-9e0e-1872b49d9883 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:02.436737 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d477c1-dd14-42f1-9e0e-1872b49d9883 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:02.442852 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 370/716] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:00:02 2022] GET /identity/v3/role_assignments?user.id=a8d2cd93042c4583b63afe629d47fce0&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:02.696175 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5d135-4d24-4aa2-8e50-aa6efcd8d439 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:02.823585 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5d135-4d24-4aa2-8e50-aa6efcd8d439 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:02.850758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:02.862288 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a8fc3-be99-4d3d-8444-776e1e599dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:02.862935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a8fc3-be99-4d3d-8444-776e1e599dcf None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:02.863217 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a8fc3-be99-4d3d-8444-776e1e599dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:02.863613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a8fc3-be99-4d3d-8444-776e1e599dcf None admin] PATH_INFO: `/v3/services/network` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:02.864779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a8fc3-be99-4d3d-8444-776e1e599dcf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:02.866188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a8fc3-be99-4d3d-8444-776e1e599dcf None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:02.875256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-151a8fc3-be99-4d3d-8444-776e1e599dcf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:00:02.875256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:02.875256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:02.875256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:02.875256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:02.875256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:02.875256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:02.875256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:02.875256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:00:02.875256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:02.875256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:02.875256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:02.875256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:02.875256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:02.875256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:02.875256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:02.875256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:02.876278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:02.876278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:02.876278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:02.876278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:02.876278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:02.876278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:02.876278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:02.876278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:02.876278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:02.876278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:02.876278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:02.876278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:02.876278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:02.876278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:02.876278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:02.876278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:02.877355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:02.877355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:02.877355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:02.877355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:02.877355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:02.877355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:02.877355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:02.877355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:02.877355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:02.877355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:02.877355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:02.877355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:02.877355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:00:02.877355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:02.877355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 347/717] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:02 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:02.884137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d477c1-dd14-42f1-9e0e-1872b49d9883 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:02.989896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d477c1-dd14-42f1-9e0e-1872b49d9883 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:03.010441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:03.025453 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d696d2-a610-4cb8-9ae0-2529c02cfbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:03.026074 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d696d2-a610-4cb8-9ae0-2529c02cfbe6 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:03.026355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d696d2-a610-4cb8-9ae0-2529c02cfbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:03.026631 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d696d2-a610-4cb8-9ae0-2529c02cfbe6 None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:03.027271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d696d2-a610-4cb8-9ae0-2529c02cfbe6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:03.027605 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d696d2-a610-4cb8-9ae0-2529c02cfbe6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:03.028999 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d696d2-a610-4cb8-9ae0-2529c02cfbe6 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:03.033691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 371/718] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 23 07:00:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:03.039816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a8fc3-be99-4d3d-8444-776e1e599dcf None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:03.147729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a8fc3-be99-4d3d-8444-776e1e599dcf None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:03.195905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee062311-be54-4cf6-9a1a-42e469bbda97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:03.196577 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee062311-be54-4cf6-9a1a-42e469bbda97 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:03.196861 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee062311-be54-4cf6-9a1a-42e469bbda97 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:03.197139 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee062311-be54-4cf6-9a1a-42e469bbda97 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:03.197881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee062311-be54-4cf6-9a1a-42e469bbda97 None admin] RBAC: Adding query filter params (type=network) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:03.198188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee062311-be54-4cf6-9a1a-42e469bbda97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:03.199619 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee062311-be54-4cf6-9a1a-42e469bbda97 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:03.207292 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 348/719] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 23 07:00:03 2022] GET /identity/v3/services?type=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:03.633895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e81ea-39d1-45f5-ac11-66341a2c93d0 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:03.634299 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e81ea-39d1-45f5-ac11-66341a2c93d0 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:03.634478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e81ea-39d1-45f5-ac11-66341a2c93d0 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:03.636113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 372/720] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:03.642785 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a02f8f-6ad8-4bd7-8f8a-9872e1266752 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:03.643244 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a02f8f-6ad8-4bd7-8f8a-9872e1266752 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:03.643322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a02f8f-6ad8-4bd7-8f8a-9872e1266752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:03.674787 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-69a02f8f-6ad8-4bd7-8f8a-9872e1266752 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:03.676130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-69a02f8f-6ad8-4bd7-8f8a-9872e1266752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:03.738487 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:03.765818 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 349/721] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:00:04.104463 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac8d44-3d69-4411-954c-519d2b6f4e71 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:04.104966 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac8d44-3d69-4411-954c-519d2b6f4e71 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:04.104966 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac8d44-3d69-4411-954c-519d2b6f4e71 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:04.106509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 373/722] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:04.111897 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adba50-939d-4d24-ab57-1d49b126a3a5 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:04.112181 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adba50-939d-4d24-ab57-1d49b126a3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:04.112454 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adba50-939d-4d24-ab57-1d49b126a3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:04.139541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-11adba50-939d-4d24-ab57-1d49b126a3a5 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:04.140751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-11adba50-939d-4d24-ab57-1d49b126a3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:04.238386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac8d44-3d69-4411-954c-519d2b6f4e71 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:04.240360 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 350/723] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:00:04.387866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac8d44-3d69-4411-954c-519d2b6f4e71 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:04.436126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc772a-1097-4f81-91a1-b60d68353340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:04.437136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc772a-1097-4f81-91a1-b60d68353340 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:04.437630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc772a-1097-4f81-91a1-b60d68353340 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:04.438090 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc772a-1097-4f81-91a1-b60d68353340 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:04.458376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc772a-1097-4f81-91a1-b60d68353340 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:00:04.463832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-78cc772a-1097-4f81-91a1-b60d68353340 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:00:04.463832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:04.463832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:04.463832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:04.463832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:04.463832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:04.463832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:04.463832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:04.463832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:04.463832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:04.463832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:04.463832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:04.463832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:00:04.463832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:00:04.463832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:00:04.463832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:00:04.464730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:00:04.464730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:00:04.464730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:00:04.464730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:00:04.464730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:04.464730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:04.464730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:04.464730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:04.464730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:04.464730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:04.464730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:04.464730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:04.464730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:04.464730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:04.464730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:04.464730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:04.465560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:04.465560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:04.465560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:04.465560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:04.465560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:00:04.465560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:00:04.465560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:00:04.465560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:00:04.465560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:00:04.465560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:00:04.465560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:00:04.465560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:04.468136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 374/724] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:00:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:04.474243 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adba50-939d-4d24-ab57-1d49b126a3a5 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:04.597255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc772a-1097-4f81-91a1-b60d68353340 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:04.638033 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adba50-939d-4d24-ab57-1d49b126a3a5 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:04.666549 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:04.667173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:04.708166 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e1520-26b5-40a0-9c2d-b94533f93291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:04.708787 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e1520-26b5-40a0-9c2d-b94533f93291 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:04.709064 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e1520-26b5-40a0-9c2d-b94533f93291 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:04.709335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e1520-26b5-40a0-9c2d-b94533f93291 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:04.710076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e1520-26b5-40a0-9c2d-b94533f93291 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:04.710380 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e1520-26b5-40a0-9c2d-b94533f93291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:04.711794 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e1520-26b5-40a0-9c2d-b94533f93291 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:04.731545 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 351/725] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 07:00:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:04.735612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc772a-1097-4f81-91a1-b60d68353340 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:04.736392 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e1520-26b5-40a0-9c2d-b94533f93291 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:04.773912 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230175e-444b-4d57-823d-00d3ff8d4ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:04.774544 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230175e-444b-4d57-823d-00d3ff8d4ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:04.774824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230175e-444b-4d57-823d-00d3ff8d4ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:04.775091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230175e-444b-4d57-823d-00d3ff8d4ed5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:04.785130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230175e-444b-4d57-823d-00d3ff8d4ed5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:00:04.790232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-b230175e-444b-4d57-823d-00d3ff8d4ed5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:00:04.790232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:04.790232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:04.790232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:04.790232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:04.790232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:04.790232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:04.790232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:04.790232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:04.790232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:04.790232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:04.790232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:04.790232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:00:04.790232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:00:04.790232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:00:04.790232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:00:04.791188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:00:04.791188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:00:04.791188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:00:04.791188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:00:04.791188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:04.791188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:04.791188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:04.791188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:04.791188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:04.791188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:04.791188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:04.791188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:04.791188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:04.791188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:04.791188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:04.791188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:04.792619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:04.792619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:04.792619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:04.792619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:04.792619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:00:04.792619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:00:04.792619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:00:04.792619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:00:04.792619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:00:04.792619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:00:04.792619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:00:04.792619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:04.792619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 375/726] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 07:00:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:04.792619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace842f7-387e-4a77-9c9c-7a98ca904980 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:04.793395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace842f7-387e-4a77-9c9c-7a98ca904980 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:04.793395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace842f7-387e-4a77-9c9c-7a98ca904980 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:04.794453 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 376/727] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:04.799648 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf58362-1615-49cd-8898-3fe4f090a680 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:04.799887 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf58362-1615-49cd-8898-3fe4f090a680 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:04.800146 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf58362-1615-49cd-8898-3fe4f090a680 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:04.802044 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 377/728] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:04.802232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf58362-1615-49cd-8898-3fe4f090a680 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:04.850981 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e1520-26b5-40a0-9c2d-b94533f93291 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:04.885965 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f2689-ae1c-4ec7-bf37-feb162c45764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:04.886600 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f2689-ae1c-4ec7-bf37-feb162c45764 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:04.886862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f2689-ae1c-4ec7-bf37-feb162c45764 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:04.887127 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f2689-ae1c-4ec7-bf37-feb162c45764 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:04.898404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f2689-ae1c-4ec7-bf37-feb162c45764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:04.899863 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f2689-ae1c-4ec7-bf37-feb162c45764 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:04.907452 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf58362-1615-49cd-8898-3fe4f090a680 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:04.910758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:04.914276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-066f2689-ae1c-4ec7-bf37-feb162c45764 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:04.914276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:04.914276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:04.914276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:04.914276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:04.914276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:04.914276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:04.914276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:04.914276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:04.914276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:04.914276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:04.914276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:04.914276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:04.914276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:04.914276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:04.914276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:04.915786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:04.915786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:04.915786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:04.915786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:04.915786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:04.915786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:04.915786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:04.915786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:04.915786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:04.915786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:04.915786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:04.915786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:04.915786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:04.915786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:04.915786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:04.915786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:04.915786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:04.916734 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:04.916734 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:04.916734 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:04.916734 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 352/729] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:04.917242 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38e69a-0d01-4b6c-a9d4-c1344c31863a None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:04.917454 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38e69a-0d01-4b6c-a9d4-c1344c31863a None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:04.917703 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38e69a-0d01-4b6c-a9d4-c1344c31863a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:04.938229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-fd38e69a-0d01-4b6c-a9d4-c1344c31863a None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:04.939597 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-fd38e69a-0d01-4b6c-a9d4-c1344c31863a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:04.953126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513aa63-b15f-43b1-9728-1cd8c417d927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:04.953740 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513aa63-b15f-43b1-9728-1cd8c417d927 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:04.954017 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513aa63-b15f-43b1-9728-1cd8c417d927 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:04.954285 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513aa63-b15f-43b1-9728-1cd8c417d927 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:04.955068 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513aa63-b15f-43b1-9728-1cd8c417d927 None admin] RBAC: Adding query filter params (name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:04.955369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513aa63-b15f-43b1-9728-1cd8c417d927 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:04.956888 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513aa63-b15f-43b1-9728-1cd8c417d927 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:04.961974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:04.967530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 378/730] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 07:00:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:04.973197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c91b6-aad0-4f82-aec4-442bfe063740 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:04.973720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c91b6-aad0-4f82-aec4-442bfe063740 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:04.974394 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c91b6-aad0-4f82-aec4-442bfe063740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:04.993039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-ef3c91b6-aad0-4f82-aec4-442bfe063740 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:04.994260 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ef3c91b6-aad0-4f82-aec4-442bfe063740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:05.014639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 353/731] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:00:05.015492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38e69a-0d01-4b6c-a9d4-c1344c31863a None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:05.091002 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 379/732] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:00:05.092344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c91b6-aad0-4f82-aec4-442bfe063740 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:05.124773 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38e69a-0d01-4b6c-a9d4-c1344c31863a None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:05.163363 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e9526-7c9d-428e-a329-e0d162ce6380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:05.164017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e9526-7c9d-428e-a329-e0d162ce6380 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:05.164396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e9526-7c9d-428e-a329-e0d162ce6380 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:05.164743 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e9526-7c9d-428e-a329-e0d162ce6380 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:05.165604 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e9526-7c9d-428e-a329-e0d162ce6380 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:05.165973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e9526-7c9d-428e-a329-e0d162ce6380 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:05.168038 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e9526-7c9d-428e-a329-e0d162ce6380 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:05.183280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 354/733] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:05.188243 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e9526-7c9d-428e-a329-e0d162ce6380 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:05.204467 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:05.212591 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:05.212910 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c91b6-aad0-4f82-aec4-442bfe063740 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:05.251601 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a018a-0d5d-41bf-a8a8-1aced609b8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:05.252593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a018a-0d5d-41bf-a8a8-1aced609b8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:05.253096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a018a-0d5d-41bf-a8a8-1aced609b8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:05.253643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a018a-0d5d-41bf-a8a8-1aced609b8d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:05.265998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a018a-0d5d-41bf-a8a8-1aced609b8d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:05.268049 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a018a-0d5d-41bf-a8a8-1aced609b8d4 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:05.284209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-735a018a-0d5d-41bf-a8a8-1aced609b8d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:05.284209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:05.284209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:05.284209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:05.284209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:05.284209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:05.284209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:05.284209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:05.284209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:05.284209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:05.284209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:05.284209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:05.284209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:05.284209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:05.284209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:05.284209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:05.285252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:05.285252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:05.285252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:05.285252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:05.285252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:05.285252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:05.285252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:05.285252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:05.285252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:05.285252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:05.285252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:05.285252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:05.285252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:05.285252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:05.285252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:05.285252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:05.285252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:05.286745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:05.286745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:05.286745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:05.289465 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 380/734] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:05.295506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a018a-0d5d-41bf-a8a8-1aced609b8d4 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:05.305626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e9526-7c9d-428e-a329-e0d162ce6380 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:05.347525 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5829de0-aa48-4ca8-9376-e7eee591428d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:05.349443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5829de0-aa48-4ca8-9376-e7eee591428d None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:05.349722 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5829de0-aa48-4ca8-9376-e7eee591428d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:05.349993 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5829de0-aa48-4ca8-9376-e7eee591428d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:05.360490 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5829de0-aa48-4ca8-9376-e7eee591428d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:05.362620 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5829de0-aa48-4ca8-9376-e7eee591428d None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:05.377544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-c5829de0-aa48-4ca8-9376-e7eee591428d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:00:05.377544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:05.377544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:05.377544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:05.377544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:05.377544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:05.377544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:05.377544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:05.377544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:05.377544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:05.377544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:05.377544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:05.377544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:00:05.377544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:00:05.377544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:00:05.377544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:00:05.378712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:05.378712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:05.378712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:00:05.378712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:05.378712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:00:05.378712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:05.378712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:05.378712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:05.378712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:05.378712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:05.378712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:05.378712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:05.378712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:05.378712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:05.378712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:05.378712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:05.378712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:05.378712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:05.379905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:05.379905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:05.379905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:00:05.379905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:00:05.379905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:00:05.379905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:00:05.379905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:00:05.379905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:00:05.379905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:00:05.379905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:05.382092 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 355/735] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:00:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:05.392326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5829de0-aa48-4ca8-9376-e7eee591428d None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:05.413218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a018a-0d5d-41bf-a8a8-1aced609b8d4 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:05.460197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c3e2b-1a8a-4092-92b4-6c65af4bf7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:05.460742 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c3e2b-1a8a-4092-92b4-6c65af4bf7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:05.461024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c3e2b-1a8a-4092-92b4-6c65af4bf7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:05.461313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c3e2b-1a8a-4092-92b4-6c65af4bf7e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:05.462011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c3e2b-1a8a-4092-92b4-6c65af4bf7e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:05.462315 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c3e2b-1a8a-4092-92b4-6c65af4bf7e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:05.463736 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c3e2b-1a8a-4092-92b4-6c65af4bf7e2 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:05.480006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 381/736] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:05.481270 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c3e2b-1a8a-4092-92b4-6c65af4bf7e2 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:05.495822 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:05.500881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:05.527980 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5829de0-aa48-4ca8-9376-e7eee591428d None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:05.564277 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda038c1-84f7-4804-8eb5-55992e3e0e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:05.564890 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda038c1-84f7-4804-8eb5-55992e3e0e04 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:05.565164 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda038c1-84f7-4804-8eb5-55992e3e0e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:05.565435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda038c1-84f7-4804-8eb5-55992e3e0e04 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:05.566161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda038c1-84f7-4804-8eb5-55992e3e0e04 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:05.566458 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda038c1-84f7-4804-8eb5-55992e3e0e04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:05.568202 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda038c1-84f7-4804-8eb5-55992e3e0e04 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:05.586351 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 356/737] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Aug 23 07:00:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:05.587164 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda038c1-84f7-4804-8eb5-55992e3e0e04 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:05.592690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c3e2b-1a8a-4092-92b4-6c65af4bf7e2 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:05.627979 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c6dac-8c1f-4002-bcf4-dd2905fac640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:05.628589 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c6dac-8c1f-4002-bcf4-dd2905fac640 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:05.628866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c6dac-8c1f-4002-bcf4-dd2905fac640 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:05.629233 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c6dac-8c1f-4002-bcf4-dd2905fac640 None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:05.630103 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c6dac-8c1f-4002-bcf4-dd2905fac640 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:05.631528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c6dac-8c1f-4002-bcf4-dd2905fac640 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:05.655024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 382/738] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 23 07:00:05 2022] POST /identity/v3/services => generated 251 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:00:05.655024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc1d13-39f7-4350-9cec-28514edfc134 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:05.655024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc1d13-39f7-4350-9cec-28514edfc134 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:05.655024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc1d13-39f7-4350-9cec-28514edfc134 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:05.655024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 383/739] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:05.655024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc1d13-39f7-4350-9cec-28514edfc134 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:05.706411 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda038c1-84f7-4804-8eb5-55992e3e0e04 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:05.741069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:05.750555 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:05.757889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc058d-a576-4a6b-affd-c0c4cc8b6123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:05.758583 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc058d-a576-4a6b-affd-c0c4cc8b6123 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:05.758861 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc058d-a576-4a6b-affd-c0c4cc8b6123 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:05.759131 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc058d-a576-4a6b-affd-c0c4cc8b6123 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:05.767335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc058d-a576-4a6b-affd-c0c4cc8b6123 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:05.768979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc058d-a576-4a6b-affd-c0c4cc8b6123 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:05.782683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-37dc058d-a576-4a6b-affd-c0c4cc8b6123 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:00:05.782683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:05.782683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:05.782683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:05.782683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:05.782683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:05.782683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:05.782683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:05.782683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:05.782683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:05.782683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:05.782683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:05.782683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:00:05.782683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:00:05.782683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:00:05.782683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:00:05.783988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:05.783988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:05.783988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:00:05.783988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:05.783988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:00:05.783988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:05.783988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:05.783988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:05.783988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:05.783988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:05.783988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:05.783988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:05.783988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:05.783988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:05.783988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:05.783988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:05.783988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:05.783988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:05.784948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:05.784948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:05.784948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:00:05.784948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:00:05.784948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:00:05.784948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:00:05.784948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:00:05.784948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:00:05.784948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:00:05.784948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:05.785569 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc1d13-39f7-4350-9cec-28514edfc134 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:05.785646 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 357/740] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 23 07:00:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:05.785646 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc058d-a576-4a6b-affd-c0c4cc8b6123 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:05.829341 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb04f2-f1fd-4b37-bd3c-93dd0fa674b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:05.829984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb04f2-f1fd-4b37-bd3c-93dd0fa674b6 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:05.830259 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb04f2-f1fd-4b37-bd3c-93dd0fa674b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:05.830528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb04f2-f1fd-4b37-bd3c-93dd0fa674b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:05.843882 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb04f2-f1fd-4b37-bd3c-93dd0fa674b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:00:05.850984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-29cb04f2-f1fd-4b37-bd3c-93dd0fa674b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:00:05.850984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:05.850984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:05.850984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:05.850984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:05.850984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:05.850984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:05.850984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:05.850984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:05.850984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:05.850984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:05.850984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:05.850984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:00:05.850984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:00:05.850984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:00:05.850984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:00:05.852096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:00:05.852096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:00:05.852096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:00:05.852096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:00:05.852096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:05.852096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:05.852096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:05.852096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:05.852096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:05.852096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:05.852096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:05.852096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:05.852096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:05.852096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:05.852096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:05.852096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:05.852961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:05.852961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:05.852961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:05.852961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:05.852961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:00:05.852961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:00:05.852961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:00:05.852961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:00:05.852961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:00:05.852961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:00:05.852961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:00:05.852961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:05.852961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 384/741] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:00:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:05.853664 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310e149-bef8-463d-abf8-8aaeb196af34 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:05.853739 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310e149-bef8-463d-abf8-8aaeb196af34 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:05.854714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310e149-bef8-463d-abf8-8aaeb196af34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:05.875031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-c310e149-bef8-463d-abf8-8aaeb196af34 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:05.876325 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c310e149-bef8-463d-abf8-8aaeb196af34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:05.908611 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc058d-a576-4a6b-affd-c0c4cc8b6123 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:05.944485 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265c082-649c-4c45-bf8b-bd08441efd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:05.945195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265c082-649c-4c45-bf8b-bd08441efd94 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:05.945481 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265c082-649c-4c45-bf8b-bd08441efd94 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:05.945755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265c082-649c-4c45-bf8b-bd08441efd94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:05.948308 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 385/742] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:00:05.948966 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310e149-bef8-463d-abf8-8aaeb196af34 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:06.021006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:06.023341 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:06.036475 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265c082-649c-4c45-bf8b-bd08441efd94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:06.039085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265c082-649c-4c45-bf8b-bd08441efd94 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:06.074371 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310e149-bef8-463d-abf8-8aaeb196af34 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:06.113009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aea60d-7366-4d13-bda2-024d2d293634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:06.114120 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aea60d-7366-4d13-bda2-024d2d293634 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:06.114800 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aea60d-7366-4d13-bda2-024d2d293634 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:06.115192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aea60d-7366-4d13-bda2-024d2d293634 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:06.116982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aea60d-7366-4d13-bda2-024d2d293634 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:06.117295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aea60d-7366-4d13-bda2-024d2d293634 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:06.118760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aea60d-7366-4d13-bda2-024d2d293634 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:06.137650 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 386/743] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Aug 23 07:00:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:06.138506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aea60d-7366-4d13-bda2-024d2d293634 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:06.180991 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 358/744] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:00:05 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:00:06.182436 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265c082-649c-4c45-bf8b-bd08441efd94 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:06.263135 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aea60d-7366-4d13-bda2-024d2d293634 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:06.280211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:06.292943 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:06.301034 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1a3ba-14b1-4535-b3c6-5bf5371fae12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:06.301737 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1a3ba-14b1-4535-b3c6-5bf5371fae12 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:06.302133 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1a3ba-14b1-4535-b3c6-5bf5371fae12 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:06.302411 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1a3ba-14b1-4535-b3c6-5bf5371fae12 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:06.302837 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265c082-649c-4c45-bf8b-bd08441efd94 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:06.303085 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1a3ba-14b1-4535-b3c6-5bf5371fae12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:06.303406 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1a3ba-14b1-4535-b3c6-5bf5371fae12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:06.304802 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1a3ba-14b1-4535-b3c6-5bf5371fae12 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:06.313406 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 387/745] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 07:00:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:06.314216 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1a3ba-14b1-4535-b3c6-5bf5371fae12 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:06.344869 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3cf88e-c378-4670-99ed-ba6e0c1423ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:06.345481 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3cf88e-c378-4670-99ed-ba6e0c1423ba None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:06.345758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3cf88e-c378-4670-99ed-ba6e0c1423ba None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:06.346028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3cf88e-c378-4670-99ed-ba6e0c1423ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:06.416919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3cf88e-c378-4670-99ed-ba6e0c1423ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:06.418407 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3cf88e-c378-4670-99ed-ba6e0c1423ba None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:06.419306 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1a3ba-14b1-4535-b3c6-5bf5371fae12 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:06.456527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49323b-043b-4f7c-b1e0-7c9fb8ba36d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:06.457868 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49323b-043b-4f7c-b1e0-7c9fb8ba36d0 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:06.457868 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49323b-043b-4f7c-b1e0-7c9fb8ba36d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:06.457868 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49323b-043b-4f7c-b1e0-7c9fb8ba36d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:06.476725 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49323b-043b-4f7c-b1e0-7c9fb8ba36d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:06.478201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49323b-043b-4f7c-b1e0-7c9fb8ba36d0 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:06.493505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-5f49323b-043b-4f7c-b1e0-7c9fb8ba36d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:06.493505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:06.493505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:06.493505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:06.493505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:06.493505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:06.493505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:06.493505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:06.493505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:06.493505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:06.493505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:06.493505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:06.493505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:06.493505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:06.493505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:06.493505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:06.494476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:06.494476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:06.494476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:06.494476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:06.494476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:06.494476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:06.494476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:06.494476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:06.494476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:06.494476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:06.494476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:06.494476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:06.494476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:06.494476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:06.494476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:06.494476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:06.494476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:06.495462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:06.495462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:06.495462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:06.495462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 388/746] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:06.495462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49323b-043b-4f7c-b1e0-7c9fb8ba36d0 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:06.527198 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:06.531261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:06.544640 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 359/747] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:00:06 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:00:06.544640 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3cf88e-c378-4670-99ed-ba6e0c1423ba None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:06.618518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49323b-043b-4f7c-b1e0-7c9fb8ba36d0 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:06.659498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bd87c-93bc-480b-ad00-dccab5c1d941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:06.660335 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bd87c-93bc-480b-ad00-dccab5c1d941 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:06.660810 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bd87c-93bc-480b-ad00-dccab5c1d941 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:06.661245 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bd87c-93bc-480b-ad00-dccab5c1d941 None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:06.661753 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3cf88e-c378-4670-99ed-ba6e0c1423ba None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:06.662192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7bd87c-93bc-480b-ad00-dccab5c1d941 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:06.663750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7bd87c-93bc-480b-ad00-dccab5c1d941 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:06.680974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 389/748] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 23 07:00:06 2022] POST /identity/v3/services => generated 240 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:00:06.682095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bd87c-93bc-480b-ad00-dccab5c1d941 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:06.716372 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ef457-f5e7-47bb-92f7-f92341f8fcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:06.717022 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ef457-f5e7-47bb-92f7-f92341f8fcc0 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:06.717301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ef457-f5e7-47bb-92f7-f92341f8fcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:06.717577 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ef457-f5e7-47bb-92f7-f92341f8fcc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:06.732625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ef457-f5e7-47bb-92f7-f92341f8fcc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:06.734605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ef457-f5e7-47bb-92f7-f92341f8fcc0 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:06.756685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-521ef457-f5e7-47bb-92f7-f92341f8fcc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:06.756685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:06.756685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:06.756685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:06.756685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:06.756685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:06.756685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:06.756685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:06.756685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:06.756685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:06.756685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:06.756685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:06.756685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:06.756685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:06.756685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:06.756685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:06.757925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:06.757925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:06.757925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:06.757925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:06.757925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:06.757925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:06.757925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:06.757925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:06.757925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:06.757925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:06.757925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:06.757925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:06.757925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:06.757925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:06.757925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:06.757925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:06.757925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:06.758908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:06.758908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:06.758908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:06.758908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 360/749] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:06.759141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ef457-f5e7-47bb-92f7-f92341f8fcc0 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:06.807039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bd87c-93bc-480b-ad00-dccab5c1d941 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:06.819821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:06.821316 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:06.855063 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b3bd8-753d-47a4-acbc-5aaaf38aad91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:06.856986 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b3bd8-753d-47a4-acbc-5aaaf38aad91 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:06.857447 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b3bd8-753d-47a4-acbc-5aaaf38aad91 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:06.857909 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b3bd8-753d-47a4-acbc-5aaaf38aad91 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:06.858756 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b3bd8-753d-47a4-acbc-5aaaf38aad91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:06.859206 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b3bd8-753d-47a4-acbc-5aaaf38aad91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:06.860987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b3bd8-753d-47a4-acbc-5aaaf38aad91 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:06.876213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 390/750] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:06.876882 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b3bd8-753d-47a4-acbc-5aaaf38aad91 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:06.900137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ef457-f5e7-47bb-92f7-f92341f8fcc0 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:06.942335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a282d7a-3cf7-4455-88dc-286b2b56ca60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:06.943972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a282d7a-3cf7-4455-88dc-286b2b56ca60 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:06.944721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a282d7a-3cf7-4455-88dc-286b2b56ca60 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:06.945191 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a282d7a-3cf7-4455-88dc-286b2b56ca60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:06.960034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a282d7a-3cf7-4455-88dc-286b2b56ca60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:06.963022 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a282d7a-3cf7-4455-88dc-286b2b56ca60 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:06.982817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-6a282d7a-3cf7-4455-88dc-286b2b56ca60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:06.982817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:06.982817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:06.982817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:06.982817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:06.982817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:06.982817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:06.982817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:06.982817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:06.982817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:06.982817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:06.982817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:06.982817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:06.982817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:06.982817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:06.982817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:06.983894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:06.983894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:06.983894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:06.983894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:06.983894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:06.983894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:06.983894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:06.983894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:06.983894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:06.983894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:06.983894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:06.983894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:06.983894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:06.983894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:06.983894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:06.983894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:06.983894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:06.984808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:06.984808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:06.984808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:06.984976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 361/751] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:06.991128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a282d7a-3cf7-4455-88dc-286b2b56ca60 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:07.002568 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b3bd8-753d-47a4-acbc-5aaaf38aad91 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:07.040799 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ebe1a-c9ae-4309-ab46-c321e030f2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:07.041410 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ebe1a-c9ae-4309-ab46-c321e030f2f3 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:07.041677 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ebe1a-c9ae-4309-ab46-c321e030f2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:07.041944 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ebe1a-c9ae-4309-ab46-c321e030f2f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:07.042586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ebe1a-c9ae-4309-ab46-c321e030f2f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:07.042879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ebe1a-c9ae-4309-ab46-c321e030f2f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:07.044280 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ebe1a-c9ae-4309-ab46-c321e030f2f3 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:07.056424 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 391/752] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:07.057419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ebe1a-c9ae-4309-ab46-c321e030f2f3 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:07.070045 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:07.105797 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:07.117259 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a282d7a-3cf7-4455-88dc-286b2b56ca60 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:07.156246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21922e8e-8980-49bd-83e8-4272ef279f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:07.157237 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21922e8e-8980-49bd-83e8-4272ef279f36 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:07.157715 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21922e8e-8980-49bd-83e8-4272ef279f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:07.158208 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21922e8e-8980-49bd-83e8-4272ef279f36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:07.158542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ebe1a-c9ae-4309-ab46-c321e030f2f3 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:07.166740 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21922e8e-8980-49bd-83e8-4272ef279f36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:07.168680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21922e8e-8980-49bd-83e8-4272ef279f36 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:07.181941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-21922e8e-8980-49bd-83e8-4272ef279f36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:07.181941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:07.181941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:07.181941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:07.181941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:07.181941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:07.181941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:07.181941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:07.181941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:07.181941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:07.181941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:07.181941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:07.181941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:00:07.181941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:00:07.181941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:00:07.182902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:00:07.182902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:07.182902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:07.182902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:07.182902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:07.182902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:07.182902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:07.182902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:07.182902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:07.182902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:07.182902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:07.182902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:07.182902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:07.182902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:07.182902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:07.182902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:07.182902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:07.183821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:00:07.183821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:00:07.183821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:00:07.183821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:00:07.183821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:00:07.183821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:00:07.183821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:07.183821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:07.191059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 362/753] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:07.192656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21922e8e-8980-49bd-83e8-4272ef279f36 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:07.194806 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af64f4-d370-4386-bd65-9b6787c1dc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:07.195623 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af64f4-d370-4386-bd65-9b6787c1dc90 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:07.195834 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af64f4-d370-4386-bd65-9b6787c1dc90 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:07.196047 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af64f4-d370-4386-bd65-9b6787c1dc90 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:07.196694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af64f4-d370-4386-bd65-9b6787c1dc90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:07.196994 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af64f4-d370-4386-bd65-9b6787c1dc90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:07.199193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af64f4-d370-4386-bd65-9b6787c1dc90 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:07.212016 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 392/754] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:07.212869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af64f4-d370-4386-bd65-9b6787c1dc90 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:07.282121 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:07.313681 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af64f4-d370-4386-bd65-9b6787c1dc90 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:07.318699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21922e8e-8980-49bd-83e8-4272ef279f36 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:07.348534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d7bf3-f06c-4c12-9ccd-af76dbf629cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:07.349219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d7bf3-f06c-4c12-9ccd-af76dbf629cc None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:07.349497 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d7bf3-f06c-4c12-9ccd-af76dbf629cc None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:07.349775 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d7bf3-f06c-4c12-9ccd-af76dbf629cc None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:07.350465 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d7bf3-f06c-4c12-9ccd-af76dbf629cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:07.350765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d7bf3-f06c-4c12-9ccd-af76dbf629cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:07.352226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d7bf3-f06c-4c12-9ccd-af76dbf629cc None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:07.359182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f355f2-9823-40d6-bd4b-38916f53e957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:07.360129 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f355f2-9823-40d6-bd4b-38916f53e957 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:07.360702 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f355f2-9823-40d6-bd4b-38916f53e957 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:07.361171 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f355f2-9823-40d6-bd4b-38916f53e957 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:07.368758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 393/755] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:00:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:07.369690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d7bf3-f06c-4c12-9ccd-af76dbf629cc None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:07.370496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f355f2-9823-40d6-bd4b-38916f53e957 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:07.372196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f355f2-9823-40d6-bd4b-38916f53e957 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:07.386151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-e0f355f2-9823-40d6-bd4b-38916f53e957 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:00:07.386151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:07.386151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:07.386151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:07.386151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:07.386151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:07.386151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:07.386151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:07.386151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:07.386151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:07.386151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:07.386151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:07.386151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:00:07.386151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:00:07.386151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:00:07.386151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:00:07.387230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:07.387230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:07.387230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:00:07.387230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:07.387230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:00:07.387230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:07.387230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:07.387230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:07.387230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:07.387230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:07.387230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:07.387230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:07.387230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:07.387230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:07.387230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:07.387230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:07.387230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:07.387230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:07.388212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:07.388212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:07.388212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:00:07.388212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:00:07.388212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:00:07.388212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:00:07.388212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:00:07.388212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:00:07.388212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:00:07.388212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:07.390915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 363/756] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Aug 23 07:00:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:07.392106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f355f2-9823-40d6-bd4b-38916f53e957 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:07.406428 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:07.471368 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d7bf3-f06c-4c12-9ccd-af76dbf629cc None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:07.500211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:07.507083 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5e285-3053-47b5-bc2b-d1dcf37d1861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:07.507691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5e285-3053-47b5-bc2b-d1dcf37d1861 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:07.507968 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5e285-3053-47b5-bc2b-d1dcf37d1861 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:07.508240 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5e285-3053-47b5-bc2b-d1dcf37d1861 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:07.510941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f355f2-9823-40d6-bd4b-38916f53e957 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:07.515239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5e285-3053-47b5-bc2b-d1dcf37d1861 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:07.516770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5e285-3053-47b5-bc2b-d1dcf37d1861 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:07.529265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-4fe5e285-3053-47b5-bc2b-d1dcf37d1861 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:07.529265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:07.529265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:07.529265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:07.529265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:07.529265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:07.529265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:07.529265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:07.529265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:07.529265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:07.529265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:07.529265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:07.529265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:00:07.529265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:00:07.529265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:00:07.530192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:00:07.530192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:07.530192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:07.530192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:07.530192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:07.530192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:07.530192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:07.530192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:07.530192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:07.530192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:07.530192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:07.530192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:07.530192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:07.530192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:07.530192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:07.530192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:07.530192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:07.531094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:00:07.531094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:00:07.531094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:00:07.531094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:00:07.531094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:00:07.531094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:00:07.531094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:07.531094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:07.534991 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 394/757] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:07.535867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5e285-3053-47b5-bc2b-d1dcf37d1861 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:07.556243 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fc651-a8bc-4ddf-b361-95c7b266413b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:07.557266 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fc651-a8bc-4ddf-b361-95c7b266413b None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:07.557763 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fc651-a8bc-4ddf-b361-95c7b266413b None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:07.558233 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fc651-a8bc-4ddf-b361-95c7b266413b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:07.559223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fc651-a8bc-4ddf-b361-95c7b266413b None admin] RBAC: Adding query filter params (role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=fd88ec8b477443af95bff9af03471adf) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:07.559827 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fc651-a8bc-4ddf-b361-95c7b266413b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:07.561548 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fc651-a8bc-4ddf-b361-95c7b266413b None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:07.568067 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 364/758] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:00:07 2022] GET /identity/v3/role_assignments?user.id=fd88ec8b477443af95bff9af03471adf&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:07.569155 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fc651-a8bc-4ddf-b361-95c7b266413b None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:07.673052 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:07.698756 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5e285-3053-47b5-bc2b-d1dcf37d1861 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:07.714059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fc651-a8bc-4ddf-b361-95c7b266413b None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:07.741220 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec9f40-d23d-4430-a0a9-e05c77a635f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:07.741942 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec9f40-d23d-4430-a0a9-e05c77a635f3 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:07.742317 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec9f40-d23d-4430-a0a9-e05c77a635f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:07.742592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec9f40-d23d-4430-a0a9-e05c77a635f3 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:07.743337 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec9f40-d23d-4430-a0a9-e05c77a635f3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:07.743691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec9f40-d23d-4430-a0a9-e05c77a635f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:07.745268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec9f40-d23d-4430-a0a9-e05c77a635f3 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:07.755877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34249b-32c9-483e-9260-2e59ca42aa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:07.756505 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34249b-32c9-483e-9260-2e59ca42aa9a None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:07.756785 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34249b-32c9-483e-9260-2e59ca42aa9a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:07.757054 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34249b-32c9-483e-9260-2e59ca42aa9a None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:07.757760 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34249b-32c9-483e-9260-2e59ca42aa9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:07.758113 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34249b-32c9-483e-9260-2e59ca42aa9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:07.759624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34249b-32c9-483e-9260-2e59ca42aa9a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:07.765881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 395/759] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Aug 23 07:00:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:07.767397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462e5ae-37d2-44f0-bced-0f4ce98e0b54 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:07.767882 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462e5ae-37d2-44f0-bced-0f4ce98e0b54 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:07.768288 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462e5ae-37d2-44f0-bced-0f4ce98e0b54 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:07.769876 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 396/760] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:07.771925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462e5ae-37d2-44f0-bced-0f4ce98e0b54 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:07.777365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 365/761] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:00:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:07.778886 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d7dad-b9e0-43d1-8477-6064b630d4ec None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:07.779176 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d7dad-b9e0-43d1-8477-6064b630d4ec None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:07.779466 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d7dad-b9e0-43d1-8477-6064b630d4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:07.804297 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-ce1d7dad-b9e0-43d1-8477-6064b630d4ec None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:07.806961 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ce1d7dad-b9e0-43d1-8477-6064b630d4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:07.849668 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:07.897991 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 366/762] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:07.899158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d7dad-b9e0-43d1-8477-6064b630d4ec None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:07.929075 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462e5ae-37d2-44f0-bced-0f4ce98e0b54 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:07.977313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:08.021347 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa688e-e652-4d19-8aa3-cd483917eca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:08.021798 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d7dad-b9e0-43d1-8477-6064b630d4ec None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:08.021997 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa688e-e652-4d19-8aa3-cd483917eca3 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:08.022277 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa688e-e652-4d19-8aa3-cd483917eca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:08.022548 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa688e-e652-4d19-8aa3-cd483917eca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:08.043864 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa688e-e652-4d19-8aa3-cd483917eca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:08.045479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa688e-e652-4d19-8aa3-cd483917eca3 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:08.073330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39c84a-da4d-4b7a-83ef-9f45fb02bd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:08.078295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-13aa688e-e652-4d19-8aa3-cd483917eca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:08.078295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:08.078295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:08.078295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:08.078295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:08.078295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:08.078295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:08.078295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:08.078295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:08.078295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:08.078295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:08.078295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:08.078295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:08.078295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:08.078295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:08.078295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:08.079459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:08.079459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:08.079459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:08.079459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:08.079459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:08.079459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:08.079459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:08.079459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:08.079459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:08.079459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:08.079459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:08.079459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:08.079459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:08.079459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:08.079459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:08.079459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:08.079459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:08.080573 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:08.080573 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:08.080573 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:08.083448 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39c84a-da4d-4b7a-83ef-9f45fb02bd88 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:08.083448 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39c84a-da4d-4b7a-83ef-9f45fb02bd88 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:08.083448 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39c84a-da4d-4b7a-83ef-9f45fb02bd88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:08.083448 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39c84a-da4d-4b7a-83ef-9f45fb02bd88 None admin] RBAC: Adding query filter params (role.id=2d62b8b8395745c7983c6c8d100d985b, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=b3f1ae88e29f4172a7db7393d517a893) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:08.083448 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39c84a-da4d-4b7a-83ef-9f45fb02bd88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:08.086202 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39c84a-da4d-4b7a-83ef-9f45fb02bd88 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:08.087471 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 397/763] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:08.096086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa688e-e652-4d19-8aa3-cd483917eca3 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:08.098662 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 367/764] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:00:07 2022] GET /identity/v3/role_assignments?user.id=b3f1ae88e29f4172a7db7393d517a893&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2d62b8b8395745c7983c6c8d100d985b => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:08.207026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39c84a-da4d-4b7a-83ef-9f45fb02bd88 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:08.239983 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:08.242934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa688e-e652-4d19-8aa3-cd483917eca3 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:08.330480 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5a069-d936-48a4-a9c4-e05c0bae9ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:08.331129 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a069-d936-48a4-a9c4-e05c0bae9ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:08.331371 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a069-d936-48a4-a9c4-e05c0bae9ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:08.331673 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a069-d936-48a4-a9c4-e05c0bae9ab5 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:08.332331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5a069-d936-48a4-a9c4-e05c0bae9ab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:08.332845 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5a069-d936-48a4-a9c4-e05c0bae9ab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:08.334776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5a069-d936-48a4-a9c4-e05c0bae9ab5 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:08.365469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 398/765] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:08.371272 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5a069-d936-48a4-a9c4-e05c0bae9ab5 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:08.414537 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39c84a-da4d-4b7a-83ef-9f45fb02bd88 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:08.443980 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:08.453955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d57b8-61c8-4e57-9807-ecfe6f2d0054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:08.454602 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d57b8-61c8-4e57-9807-ecfe6f2d0054 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:08.454886 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d57b8-61c8-4e57-9807-ecfe6f2d0054 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:08.455165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d57b8-61c8-4e57-9807-ecfe6f2d0054 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:08.455795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d57b8-61c8-4e57-9807-ecfe6f2d0054 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:08.457672 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d57b8-61c8-4e57-9807-ecfe6f2d0054 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:08.467979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-b82d57b8-61c8-4e57-9807-ecfe6f2d0054 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:00:08.467979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:08.467979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:08.467979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:08.467979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:08.467979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:08.467979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:08.467979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:08.467979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:00:08.467979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:08.467979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:08.467979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:08.467979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:08.467979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:08.467979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:08.467979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:08.467979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:08.469017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:08.469017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:08.469017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:08.469017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:08.469017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:08.469017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:08.469017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:08.469017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:08.469017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:08.469017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:08.469017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:08.469017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:08.469017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:08.469017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:08.469017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:08.469017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:08.469940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:08.469940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:08.469940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:08.469940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:08.469940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:08.469940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:08.469940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:08.469940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:08.469940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:08.469940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:08.469940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:08.469940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:08.469940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:00:08.469940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:08.470667 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 368/766] 10.222.0.17 () {56 vars in 1129 bytes} [Tue Aug 23 07:00:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:08.479555 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d57b8-61c8-4e57-9807-ecfe6f2d0054 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:08.547205 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5a069-d936-48a4-a9c4-e05c0bae9ab5 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:08.597589 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad59ade-a9bc-4e2f-adc3-e57b20ee2a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:08.598311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad59ade-a9bc-4e2f-adc3-e57b20ee2a13 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:08.598651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad59ade-a9bc-4e2f-adc3-e57b20ee2a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:08.598926 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad59ade-a9bc-4e2f-adc3-e57b20ee2a13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:08.615948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad59ade-a9bc-4e2f-adc3-e57b20ee2a13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:08.617445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad59ade-a9bc-4e2f-adc3-e57b20ee2a13 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:08.643987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-3ad59ade-a9bc-4e2f-adc3-e57b20ee2a13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:08.643987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:08.643987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:08.643987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:08.643987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:08.643987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:08.643987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:08.643987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:08.643987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:08.643987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:08.643987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:08.643987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:08.643987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:00:08.643987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:00:08.643987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:00:08.646265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:00:08.646265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:08.646265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:08.646265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:08.646265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:08.646265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:08.646265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:08.646265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:08.646265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:08.646265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:08.646265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:08.646265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:08.646265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:08.646265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:08.646265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:08.646265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:08.646265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:08.648535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:00:08.648535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:00:08.648535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:00:08.648535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:00:08.648535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:00:08.648535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:00:08.648535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:08.648535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:08.648535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 399/767] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:08.657902 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad59ade-a9bc-4e2f-adc3-e57b20ee2a13 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:08.682427 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d57b8-61c8-4e57-9807-ecfe6f2d0054 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:08.704074 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:08.743022 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3bb41-a812-450b-a218-ee357ba6db03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:08.743979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3bb41-a812-450b-a218-ee357ba6db03 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:08.744265 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3bb41-a812-450b-a218-ee357ba6db03 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:08.744804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3bb41-a812-450b-a218-ee357ba6db03 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:08.745457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3bb41-a812-450b-a218-ee357ba6db03 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:08.745760 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3bb41-a812-450b-a218-ee357ba6db03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:08.748044 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3bb41-a812-450b-a218-ee357ba6db03 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:08.754667 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 369/768] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Aug 23 07:00:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:08.762065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3bb41-a812-450b-a218-ee357ba6db03 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:08.853127 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad59ade-a9bc-4e2f-adc3-e57b20ee2a13 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:08.871957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:08.894757 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42da70-c2f8-4ff5-89aa-aa6bf635134c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:08.895813 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42da70-c2f8-4ff5-89aa-aa6bf635134c None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:08.896090 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42da70-c2f8-4ff5-89aa-aa6bf635134c None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:08.896356 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42da70-c2f8-4ff5-89aa-aa6bf635134c None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:08.897355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42da70-c2f8-4ff5-89aa-aa6bf635134c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:08.897923 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42da70-c2f8-4ff5-89aa-aa6bf635134c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:08.899542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42da70-c2f8-4ff5-89aa-aa6bf635134c None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:08.914432 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 400/769] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:00:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:08.915920 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959d561-bc24-4296-9580-488ffe011530 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:08.916204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959d561-bc24-4296-9580-488ffe011530 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:08.916767 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959d561-bc24-4296-9580-488ffe011530 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:08.918469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 401/770] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:08.920604 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959d561-bc24-4296-9580-488ffe011530 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:08.921670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3bb41-a812-450b-a218-ee357ba6db03 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:08.964766 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78748e-aaae-484c-885e-c2c1114e6a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:08.965376 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78748e-aaae-484c-885e-c2c1114e6a41 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:08.965648 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78748e-aaae-484c-885e-c2c1114e6a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:08.965913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78748e-aaae-484c-885e-c2c1114e6a41 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:08.966544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78748e-aaae-484c-885e-c2c1114e6a41 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:08.966842 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78748e-aaae-484c-885e-c2c1114e6a41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:08.968234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78748e-aaae-484c-885e-c2c1114e6a41 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:08.974759 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 370/771] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Aug 23 07:00:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:08.976012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81989033-683b-4d6c-b046-7ead27260609 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:08.976298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81989033-683b-4d6c-b046-7ead27260609 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:08.976573 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81989033-683b-4d6c-b046-7ead27260609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:08.998878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-81989033-683b-4d6c-b046-7ead27260609 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:09.000004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-81989033-683b-4d6c-b046-7ead27260609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:09.025582 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:09.049940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959d561-bc24-4296-9580-488ffe011530 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:09.074303 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 371/772] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:09.074303 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81989033-683b-4d6c-b046-7ead27260609 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:09.101345 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767309e-d092-477b-b297-8f8d51239157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:09.107433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767309e-d092-477b-b297-8f8d51239157 None admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:09.109465 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767309e-d092-477b-b297-8f8d51239157 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:09.109786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767309e-d092-477b-b297-8f8d51239157 None admin] PATH_INFO: `/v3/projects/351a6777525c4207826d7df0d3768e99/users/a8d2cd93042c4583b63afe629d47fce0/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:09.112859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767309e-d092-477b-b297-8f8d51239157 None admin] RBAC: Authorizing `identity:create_grant(project_id=351a6777525c4207826d7df0d3768e99, user_id=a8d2cd93042c4583b63afe629d47fce0, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:09.114744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767309e-d092-477b-b297-8f8d51239157 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:09.123798 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:09.139954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 402/773] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 07:00:08 2022] PUT /identity/v3/projects/351a6777525c4207826d7df0d3768e99/users/a8d2cd93042c4583b63afe629d47fce0/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:00:09.179057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81989033-683b-4d6c-b046-7ead27260609 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:09.216446 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22492946-bc34-4dfa-bbb4-f63149a21ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:09.217121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22492946-bc34-4dfa-bbb4-f63149a21ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:09.217397 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22492946-bc34-4dfa-bbb4-f63149a21ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:09.217664 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22492946-bc34-4dfa-bbb4-f63149a21ef0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:09.218423 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22492946-bc34-4dfa-bbb4-f63149a21ef0 None admin] RBAC: Adding query filter params (interface=public, service_id=1fa594d7952349f29606ff411f55b696, region_id=RegionOne) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:09.218757 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22492946-bc34-4dfa-bbb4-f63149a21ef0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:09.220201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22492946-bc34-4dfa-bbb4-f63149a21ef0 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:09.229227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 372/774] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Aug 23 07:00:09 2022] GET /identity/v3/endpoints?service_id=1fa594d7952349f29606ff411f55b696&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:09.234812 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767309e-d092-477b-b297-8f8d51239157 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:09.343634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767309e-d092-477b-b297-8f8d51239157 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:09.379167 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4c3c7-5303-46c5-a924-c9a1b3dba067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:09.380054 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4c3c7-5303-46c5-a924-c9a1b3dba067 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:09.380421 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4c3c7-5303-46c5-a924-c9a1b3dba067 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:09.380875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4c3c7-5303-46c5-a924-c9a1b3dba067 None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:09.381543 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4c3c7-5303-46c5-a924-c9a1b3dba067 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:09.382992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4c3c7-5303-46c5-a924-c9a1b3dba067 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:09.388928 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 403/775] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Aug 23 07:00:09 2022] GET /identity/v3/services => generated 772 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:09.424915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22492946-bc34-4dfa-bbb4-f63149a21ef0 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:09.549228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22492946-bc34-4dfa-bbb4-f63149a21ef0 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:09.564255 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:09.593534 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff2121-800b-47b7-92f5-afd2a22ba825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:09.594245 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff2121-800b-47b7-92f5-afd2a22ba825 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:09.594464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff2121-800b-47b7-92f5-afd2a22ba825 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:09.594735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff2121-800b-47b7-92f5-afd2a22ba825 None admin] PATH_INFO: `/v3/services/network` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:09.595325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff2121-800b-47b7-92f5-afd2a22ba825 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:09.596813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff2121-800b-47b7-92f5-afd2a22ba825 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:09.607758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-f9ff2121-800b-47b7-92f5-afd2a22ba825 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:00:09.607758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:09.607758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:09.607758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:09.607758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:09.607758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:09.607758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:09.607758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:09.607758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:00:09.607758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:09.607758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:09.607758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:09.607758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:09.607758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:09.607758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:09.607758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:09.607758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:09.608819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:09.608819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:09.608819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:09.608819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:09.608819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:09.608819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:09.608819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:09.608819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:09.608819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:09.608819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:09.608819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:09.608819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:09.608819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:09.608819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:09.608819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:09.608819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:09.609691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:09.609691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:09.609691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:09.609691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:09.609691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:09.609691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:09.609691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:09.609691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:09.609691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:09.609691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:09.609691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:09.609691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:09.609691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:00:09.609691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:09.609691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 373/776] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:09 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:09.620586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4c3c7-5303-46c5-a924-c9a1b3dba067 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:09.624526 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:09.753372 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4c3c7-5303-46c5-a924-c9a1b3dba067 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:09.803055 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b53c8-d40f-4420-a139-d8b40fe67f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:09.803690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b53c8-d40f-4420-a139-d8b40fe67f6d None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:09.803964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b53c8-d40f-4420-a139-d8b40fe67f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:09.804233 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b53c8-d40f-4420-a139-d8b40fe67f6d None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:09.804862 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b53c8-d40f-4420-a139-d8b40fe67f6d None admin] RBAC: Adding query filter params (name=network) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:09.805164 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b53c8-d40f-4420-a139-d8b40fe67f6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:09.806530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b53c8-d40f-4420-a139-d8b40fe67f6d None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:09.810128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb0a88-7b6b-43c6-b2df-949932d30f4c None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:09.810574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb0a88-7b6b-43c6-b2df-949932d30f4c None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:09.810649 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb0a88-7b6b-43c6-b2df-949932d30f4c None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:09.812305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 374/777] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:00:09.815960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 404/778] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 23 07:00:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:09.821241 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb0a88-7b6b-43c6-b2df-949932d30f4c None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:09.821704 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2702f49-cb86-41b4-a61e-6fbceeffb8ec None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:09.821996 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2702f49-cb86-41b4-a61e-6fbceeffb8ec None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:09.822271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2702f49-cb86-41b4-a61e-6fbceeffb8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:09.847130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-c2702f49-cb86-41b4-a61e-6fbceeffb8ec None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:09.848287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c2702f49-cb86-41b4-a61e-6fbceeffb8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:09.941612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 405/779] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:09.943786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:10.014870 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb0a88-7b6b-43c6-b2df-949932d30f4c None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:10.053916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08083c26-c352-46ba-9500-cd2765e63b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:10.054534 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08083c26-c352-46ba-9500-cd2765e63b6c None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:10.054814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08083c26-c352-46ba-9500-cd2765e63b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:10.055091 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08083c26-c352-46ba-9500-cd2765e63b6c None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:10.055788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08083c26-c352-46ba-9500-cd2765e63b6c None admin] RBAC: Adding query filter params (type=network) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:10.056093 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08083c26-c352-46ba-9500-cd2765e63b6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:10.057485 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08083c26-c352-46ba-9500-cd2765e63b6c None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:10.062891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 375/780] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 23 07:00:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:10.068780 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2702f49-cb86-41b4-a61e-6fbceeffb8ec None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:10.207578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2702f49-cb86-41b4-a61e-6fbceeffb8ec None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:10.258394 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c73eb-f3a3-4c1a-92fb-b163f6566a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:10.259010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c73eb-f3a3-4c1a-92fb-b163f6566a25 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:10.259288 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c73eb-f3a3-4c1a-92fb-b163f6566a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:10.259594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c73eb-f3a3-4c1a-92fb-b163f6566a25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:10.260301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c73eb-f3a3-4c1a-92fb-b163f6566a25 None admin] RBAC: Adding query filter params (interface=public, service_id=0c492d31bbfc4fbf98441c707f2b8c51, region_id=RegionOne) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:10.260606 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c73eb-f3a3-4c1a-92fb-b163f6566a25 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:10.261978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c73eb-f3a3-4c1a-92fb-b163f6566a25 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:10.274811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 406/781] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Aug 23 07:00:10 2022] GET /identity/v3/endpoints?service_id=0c492d31bbfc4fbf98441c707f2b8c51&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:10.279919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08083c26-c352-46ba-9500-cd2765e63b6c None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:10.281493 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c73eb-f3a3-4c1a-92fb-b163f6566a25 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:10.334933 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:10.358602 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:10.402671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08083c26-c352-46ba-9500-cd2765e63b6c None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:10.404583 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c73eb-f3a3-4c1a-92fb-b163f6566a25 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:10.439932 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a2ef8-becc-4ac6-8583-fbefe6a3ae76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:10.440835 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a2ef8-becc-4ac6-8583-fbefe6a3ae76 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:10.441378 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a2ef8-becc-4ac6-8583-fbefe6a3ae76 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:10.441880 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a2ef8-becc-4ac6-8583-fbefe6a3ae76 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:10.442738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417a2ef8-becc-4ac6-8583-fbefe6a3ae76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:10.444295 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417a2ef8-becc-4ac6-8583-fbefe6a3ae76 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:10.447178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b7ca2-b706-4703-b66b-328936b89d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:10.448099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b7ca2-b706-4703-b66b-328936b89d1e None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:10.448377 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b7ca2-b706-4703-b66b-328936b89d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:10.448650 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b7ca2-b706-4703-b66b-328936b89d1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:10.455734 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 376/782] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Aug 23 07:00:10 2022] GET /identity/v3/services => generated 772 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:10.457711 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bc4f3-0f23-40ea-8cff-d9435cc25ff2 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:10.458274 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bc4f3-0f23-40ea-8cff-d9435cc25ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:10.458684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bc4f3-0f23-40ea-8cff-d9435cc25ff2 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:10.460234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 377/783] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:10.460346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b7ca2-b706-4703-b66b-328936b89d1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:00:10.465679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-059b7ca2-b706-4703-b66b-328936b89d1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:00:10.465679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:10.465679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:10.465679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:10.465679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:10.465679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:10.465679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:10.465679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:10.465679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:10.465679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:10.465679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:10.465679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:10.465679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:00:10.465679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:00:10.465679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:00:10.465679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:00:10.466874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:00:10.466874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:00:10.466874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:00:10.466874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:00:10.466874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:10.466874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:10.466874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:10.466874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:10.466874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:10.466874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:10.466874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:10.466874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:10.466874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:10.466874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:10.466874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:10.466874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:10.467837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:10.467837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:10.467837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:10.467837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:10.467837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:00:10.467837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:00:10.467837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:00:10.467837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:00:10.467837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:00:10.467837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:00:10.467837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:00:10.467837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:10.467837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 407/784] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:00:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:10.474784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bc4f3-0f23-40ea-8cff-d9435cc25ff2 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:10.478890 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7e641-849c-4973-8e93-38026aad3f98 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:10.479172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7e641-849c-4973-8e93-38026aad3f98 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:10.479472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7e641-849c-4973-8e93-38026aad3f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:10.506576 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-0ca7e641-849c-4973-8e93-38026aad3f98 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:10.507985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-0ca7e641-849c-4973-8e93-38026aad3f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:10.588194 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 408/785] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Aug 23 07:00:10.680545 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bc4f3-0f23-40ea-8cff-d9435cc25ff2 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:10.691495 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:10.726693 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b3b2f-a701-46d0-99ed-d56028c8f07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:10.727223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b3b2f-a701-46d0-99ed-d56028c8f07c None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:10.727564 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b3b2f-a701-46d0-99ed-d56028c8f07c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:10.727839 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b3b2f-a701-46d0-99ed-d56028c8f07c None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:10.728507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b3b2f-a701-46d0-99ed-d56028c8f07c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:10.728805 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b3b2f-a701-46d0-99ed-d56028c8f07c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:10.730202 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b3b2f-a701-46d0-99ed-d56028c8f07c None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:10.741018 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 378/786] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 07:00:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:10.746702 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7e641-849c-4973-8e93-38026aad3f98 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:10.804033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:10.856764 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7e641-849c-4973-8e93-38026aad3f98 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:10.911005 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3b22b-6743-49dc-b781-f6029eb12197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:10.912317 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3b22b-6743-49dc-b781-f6029eb12197 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:10.912598 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3b22b-6743-49dc-b781-f6029eb12197 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:10.912873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3b22b-6743-49dc-b781-f6029eb12197 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:10.931760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3b22b-6743-49dc-b781-f6029eb12197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:10.933251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3b22b-6743-49dc-b781-f6029eb12197 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:10.956228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-76c3b22b-6743-49dc-b781-f6029eb12197 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:10.956228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:10.956228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:10.956228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:10.956228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:10.956228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:10.956228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:10.956228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:10.956228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:10.956228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:10.956228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:10.956228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:10.956228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:10.956228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:10.956228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:10.956228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:10.957213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:10.957213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:10.957213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:10.957213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:10.957213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:10.957213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:10.957213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:10.957213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:10.957213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:10.957213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:10.957213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:10.957213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:10.957213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:10.957213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:10.957213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:10.957213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:10.957213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:10.958141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:10.958141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:10.958141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:10.958141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 409/787] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:10.964144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b3b2f-a701-46d0-99ed-d56028c8f07c None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:10.992013 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3b22b-6743-49dc-b781-f6029eb12197 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:11.206397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3b22b-6743-49dc-b781-f6029eb12197 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:11.212512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:11.255381 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b3b2f-a701-46d0-99ed-d56028c8f07c None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:11.277049 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce626ba-f393-48da-85c9-c1c94809acba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:11.278035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce626ba-f393-48da-85c9-c1c94809acba None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:11.278507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce626ba-f393-48da-85c9-c1c94809acba None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:11.279020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce626ba-f393-48da-85c9-c1c94809acba None admin] PATH_INFO: `/v3/roles/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:11.292304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1c113-5502-44ec-909a-5d678f42d192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:11.293044 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1c113-5502-44ec-909a-5d678f42d192 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:11.293327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1c113-5502-44ec-909a-5d678f42d192 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:11.293600 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1c113-5502-44ec-909a-5d678f42d192 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:11.294247 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd1c113-5502-44ec-909a-5d678f42d192 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:11.294573 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd1c113-5502-44ec-909a-5d678f42d192 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:11.299446 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd1c113-5502-44ec-909a-5d678f42d192 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:11.308321 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce626ba-f393-48da-85c9-c1c94809acba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:00:11.309512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 379/788] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:11.314239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-fce626ba-f393-48da-85c9-c1c94809acba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:00:11.314239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:11.314239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:11.314239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:11.314239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:11.314239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:11.314239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:11.314239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:11.314239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:11.314239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:11.314239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:11.314239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:11.314239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:00:11.314239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:00:11.314239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:00:11.314239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:00:11.315137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:00:11.315137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:00:11.315137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:00:11.315137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:00:11.315137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:11.315137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:11.315137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:11.315137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:11.315137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:11.315137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:11.315137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:11.315137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:11.315137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:11.315137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:11.315137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:11.315137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:11.316007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:11.316007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:11.316007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:11.316007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:11.316007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:00:11.316007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:00:11.316007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:00:11.316007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:00:11.316007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:00:11.316007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:00:11.316007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:00:11.316007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:11.319083 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 410/789] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 07:00:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:11.320094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1c113-5502-44ec-909a-5d678f42d192 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:11.324238 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce626ba-f393-48da-85c9-c1c94809acba None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:11.354190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:11.439286 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1c113-5502-44ec-909a-5d678f42d192 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:11.480102 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf1771-5676-4f7c-9ec3-f98448097248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:11.481734 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf1771-5676-4f7c-9ec3-f98448097248 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:11.483095 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf1771-5676-4f7c-9ec3-f98448097248 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:11.483717 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf1771-5676-4f7c-9ec3-f98448097248 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:11.505686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf1771-5676-4f7c-9ec3-f98448097248 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:11.508043 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf1771-5676-4f7c-9ec3-f98448097248 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:11.527158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-24cf1771-5676-4f7c-9ec3-f98448097248 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:00:11.527158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:11.527158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:11.527158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:11.527158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:11.527158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:11.527158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:11.527158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:11.527158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:11.527158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:11.527158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:11.527158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:11.527158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:00:11.527158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:00:11.527158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:00:11.527158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:00:11.528188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:11.528188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:11.528188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:00:11.528188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:11.528188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:00:11.528188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:11.528188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:11.528188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:11.528188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:11.528188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:11.528188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:11.528188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:11.528188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:11.528188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:11.528188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:11.528188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:11.528188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:11.528188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:11.529682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:11.529682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:11.529682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:00:11.529682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:00:11.529682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:00:11.529682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:00:11.529682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:00:11.529682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:00:11.529682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:00:11.529682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:11.529682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 380/790] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:00:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:11.530259 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43981f35-44af-453d-afc1-b2738079e32b None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:11.530259 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43981f35-44af-453d-afc1-b2738079e32b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:11.530259 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43981f35-44af-453d-afc1-b2738079e32b None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:11.531695 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 381/791] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:11.536743 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ff85b-7b04-4cde-8e4c-c06a981150f9 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:11.537060 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ff85b-7b04-4cde-8e4c-c06a981150f9 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:11.537296 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ff85b-7b04-4cde-8e4c-c06a981150f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:11.552419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce626ba-f393-48da-85c9-c1c94809acba None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:11.557360 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-347ff85b-7b04-4cde-8e4c-c06a981150f9 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:11.558655 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-347ff85b-7b04-4cde-8e4c-c06a981150f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:11.597221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938cad6-c29e-4ed4-9abd-f5bcbfe47d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:11.597985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938cad6-c29e-4ed4-9abd-f5bcbfe47d91 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:11.598365 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938cad6-c29e-4ed4-9abd-f5bcbfe47d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:11.599340 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938cad6-c29e-4ed4-9abd-f5bcbfe47d91 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:11.599901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938cad6-c29e-4ed4-9abd-f5bcbfe47d91 None admin] RBAC: Adding query filter params (name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:11.600318 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938cad6-c29e-4ed4-9abd-f5bcbfe47d91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:11.601790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938cad6-c29e-4ed4-9abd-f5bcbfe47d91 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:11.610631 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 411/792] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 07:00:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:11.611772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938cad6-c29e-4ed4-9abd-f5bcbfe47d91 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:11.638271 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 382/793] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:11.639397 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ff85b-7b04-4cde-8e4c-c06a981150f9 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:11.650339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:11.663768 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:11.779147 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ff85b-7b04-4cde-8e4c-c06a981150f9 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:11.779871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938cad6-c29e-4ed4-9abd-f5bcbfe47d91 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:11.818583 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8392c-fa32-4621-a914-0b4677751e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:11.819522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8392c-fa32-4621-a914-0b4677751e70 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:11.819955 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8392c-fa32-4621-a914-0b4677751e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:11.821026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8392c-fa32-4621-a914-0b4677751e70 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:11.822085 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a8392c-fa32-4621-a914-0b4677751e70 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:11.822603 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a8392c-fa32-4621-a914-0b4677751e70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:11.824377 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a8392c-fa32-4621-a914-0b4677751e70 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:11.851800 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 412/794] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Aug 23 07:00:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:11.856869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8392c-fa32-4621-a914-0b4677751e70 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:11.868843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bf4c1-b2fc-4ab3-8ffc-ac6f21247d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:11.869485 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bf4c1-b2fc-4ab3-8ffc-ac6f21247d36 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:11.869762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bf4c1-b2fc-4ab3-8ffc-ac6f21247d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:11.870031 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bf4c1-b2fc-4ab3-8ffc-ac6f21247d36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:11.891924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bf4c1-b2fc-4ab3-8ffc-ac6f21247d36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:11.893325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bf4c1-b2fc-4ab3-8ffc-ac6f21247d36 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:11.909027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-fc4bf4c1-b2fc-4ab3-8ffc-ac6f21247d36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:11.909027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:11.909027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:11.909027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:11.909027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:11.909027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:11.909027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:11.909027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:11.909027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:11.909027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:11.909027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:11.909027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:11.909027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:11.909027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:11.909027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:11.909027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:11.910115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:11.910115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:11.910115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:11.910115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:11.910115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:11.910115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:11.910115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:11.910115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:11.910115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:11.910115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:11.910115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:11.910115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:11.910115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:11.910115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:11.910115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:11.910115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:11.910115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:11.911858 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:11.911858 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:11.911858 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:11.911858 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 383/795] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:11.912131 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05a660-d72d-47f2-a3ec-e407d5d74bae None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:11.912412 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05a660-d72d-47f2-a3ec-e407d5d74bae None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:11.912685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05a660-d72d-47f2-a3ec-e407d5d74bae None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:11.914152 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 384/796] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:11.920656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c331f-119f-4e97-912d-99aa6679cafd None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:11.920943 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c331f-119f-4e97-912d-99aa6679cafd None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:11.921299 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c331f-119f-4e97-912d-99aa6679cafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:11.941343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-da7c331f-119f-4e97-912d-99aa6679cafd None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:11.943250 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-da7c331f-119f-4e97-912d-99aa6679cafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:11.970598 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:11.990633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8392c-fa32-4621-a914-0b4677751e70 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:12.015243 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:12.027560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 385/797] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:12.028469 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c331f-119f-4e97-912d-99aa6679cafd None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:12.035822 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e7e35-39ba-41ff-845a-03c449a0c308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:12.036739 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e7e35-39ba-41ff-845a-03c449a0c308 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:12.037332 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e7e35-39ba-41ff-845a-03c449a0c308 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:12.038260 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e7e35-39ba-41ff-845a-03c449a0c308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:12.052044 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e7e35-39ba-41ff-845a-03c449a0c308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:12.054248 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e7e35-39ba-41ff-845a-03c449a0c308 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:12.073530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-898e7e35-39ba-41ff-845a-03c449a0c308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:12.073530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:12.073530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:12.073530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:12.073530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:12.073530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:12.073530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:12.073530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:12.073530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:12.073530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:12.073530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:12.073530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:12.073530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:12.073530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:12.073530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:12.073530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:12.074501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:12.074501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:12.074501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:12.074501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:12.074501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:12.074501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:12.074501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:12.074501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:12.074501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:12.074501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:12.074501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:12.074501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:12.074501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:12.074501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:12.074501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:12.074501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:12.074501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:12.075391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:12.075391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:12.075391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:12.081981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 413/798] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:12.081981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e7e35-39ba-41ff-845a-03c449a0c308 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:12.142364 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c331f-119f-4e97-912d-99aa6679cafd None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:12.177923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad643b8-ce4a-4ff6-84d1-9f893416d6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:12.178889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad643b8-ce4a-4ff6-84d1-9f893416d6da None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:12.179507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad643b8-ce4a-4ff6-84d1-9f893416d6da None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:12.180160 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad643b8-ce4a-4ff6-84d1-9f893416d6da None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:12.181372 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad643b8-ce4a-4ff6-84d1-9f893416d6da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:12.182021 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad643b8-ce4a-4ff6-84d1-9f893416d6da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:12.183617 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad643b8-ce4a-4ff6-84d1-9f893416d6da None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:12.184616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e7e35-39ba-41ff-845a-03c449a0c308 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:12.196346 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 386/799] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:12.197576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad643b8-ce4a-4ff6-84d1-9f893416d6da None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:12.223053 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56802898-07d8-40de-8dba-3f8feb543ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:12.223984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56802898-07d8-40de-8dba-3f8feb543ced None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:12.224333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56802898-07d8-40de-8dba-3f8feb543ced None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:12.224641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56802898-07d8-40de-8dba-3f8feb543ced None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:12.235806 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:12.240552 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56802898-07d8-40de-8dba-3f8feb543ced None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:00:12.246095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-56802898-07d8-40de-8dba-3f8feb543ced None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:00:12.246095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:12.246095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:12.246095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:12.246095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:12.246095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:12.246095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:12.246095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:12.246095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:12.246095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:12.246095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:12.246095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:12.246095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:00:12.246095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:00:12.246095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:00:12.246095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:00:12.247035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:00:12.247035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:00:12.247035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:00:12.247035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:00:12.247035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:12.247035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:12.247035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:12.247035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:12.247035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:12.247035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:12.247035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:12.247035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:12.247035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:12.247035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:12.247035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:12.247035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:12.247900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:12.247900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:12.247900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:12.247900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:12.247900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:00:12.247900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:00:12.247900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:00:12.247900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:00:12.247900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:00:12.247900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:00:12.247900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:00:12.247900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:12.251864 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 414/800] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:00:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:12.252783 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56802898-07d8-40de-8dba-3f8feb543ced None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:12.271962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:12.315251 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad643b8-ce4a-4ff6-84d1-9f893416d6da None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:12.351977 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59bfcc-0aa4-4ae2-a19d-b7790b909813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:12.352714 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59bfcc-0aa4-4ae2-a19d-b7790b909813 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:12.352989 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59bfcc-0aa4-4ae2-a19d-b7790b909813 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:12.353248 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59bfcc-0aa4-4ae2-a19d-b7790b909813 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:12.353908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59bfcc-0aa4-4ae2-a19d-b7790b909813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:12.354202 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59bfcc-0aa4-4ae2-a19d-b7790b909813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:12.355603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59bfcc-0aa4-4ae2-a19d-b7790b909813 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:12.360211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56802898-07d8-40de-8dba-3f8feb543ced None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:12.367680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 387/801] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:12.372803 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59bfcc-0aa4-4ae2-a19d-b7790b909813 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:12.396954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309fa059-75d0-4f74-8ad9-537ad7fb82fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:12.397587 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fa059-75d0-4f74-8ad9-537ad7fb82fa None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:12.397855 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fa059-75d0-4f74-8ad9-537ad7fb82fa None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:12.398188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fa059-75d0-4f74-8ad9-537ad7fb82fa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:12.405855 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309fa059-75d0-4f74-8ad9-537ad7fb82fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:12.407304 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309fa059-75d0-4f74-8ad9-537ad7fb82fa None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:12.418531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-309fa059-75d0-4f74-8ad9-537ad7fb82fa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:00:12.418531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:12.418531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:12.418531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:12.418531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:12.418531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:12.418531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:12.418531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:12.418531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:12.418531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:12.418531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:12.418531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:12.418531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:00:12.418531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:00:12.418531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:00:12.418531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:00:12.419538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:12.419538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:12.419538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:00:12.419538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:12.419538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:00:12.419538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:12.419538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:12.419538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:12.419538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:12.419538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:12.419538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:12.419538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:12.419538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:12.419538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:12.419538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:12.419538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:12.419538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:12.419538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:12.420507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:12.420507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:12.420507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:00:12.420507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:00:12.420507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:00:12.420507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:00:12.420507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:00:12.420507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:00:12.420507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:00:12.420507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:12.420507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 415/802] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 23 07:00:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:12.420507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309fa059-75d0-4f74-8ad9-537ad7fb82fa None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:12.478474 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59bfcc-0aa4-4ae2-a19d-b7790b909813 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:12.480613 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:12.499147 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:12.515813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56a9df-b36c-443b-b72e-54b8c3a677f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:12.516455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56a9df-b36c-443b-b72e-54b8c3a677f8 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:12.516731 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56a9df-b36c-443b-b72e-54b8c3a677f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:12.516998 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56a9df-b36c-443b-b72e-54b8c3a677f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:12.517662 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56a9df-b36c-443b-b72e-54b8c3a677f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:12.517959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56a9df-b36c-443b-b72e-54b8c3a677f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:12.519346 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56a9df-b36c-443b-b72e-54b8c3a677f8 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:12.531444 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 388/803] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 07:00:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:12.531444 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56a9df-b36c-443b-b72e-54b8c3a677f8 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:12.540131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309fa059-75d0-4f74-8ad9-537ad7fb82fa None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:12.577006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35547478-a64e-4ff1-95cc-279fc5d6cb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:12.578728 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35547478-a64e-4ff1-95cc-279fc5d6cb1b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:12.579079 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35547478-a64e-4ff1-95cc-279fc5d6cb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:12.579358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35547478-a64e-4ff1-95cc-279fc5d6cb1b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:12.580011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35547478-a64e-4ff1-95cc-279fc5d6cb1b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:12.581384 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35547478-a64e-4ff1-95cc-279fc5d6cb1b None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:12.593439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-35547478-a64e-4ff1-95cc-279fc5d6cb1b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:00:12.593439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:12.593439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:12.593439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:12.593439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:12.593439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:12.593439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:12.593439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:12.593439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:00:12.593439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:12.593439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:12.593439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:12.593439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:12.593439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:12.593439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:12.593439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:12.593439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:12.594454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:12.594454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:12.594454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:12.594454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:12.594454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:12.594454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:12.594454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:12.594454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:12.594454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:12.594454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:12.594454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:12.594454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:12.594454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:12.594454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:12.594454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:12.594454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:12.595892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:12.595892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:12.595892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:12.595892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:12.595892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:12.595892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:12.595892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:12.595892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:12.595892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:12.595892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:12.595892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:12.595892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:12.595892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:00:12.595892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:12.595892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 416/804] 10.222.0.17 () {56 vars in 1129 bytes} [Tue Aug 23 07:00:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:12.596723 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35547478-a64e-4ff1-95cc-279fc5d6cb1b None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:12.651843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56a9df-b36c-443b-b72e-54b8c3a677f8 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:12.687835 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64720407-8912-4bf3-9cf6-bd5b111c64f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:12.688485 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64720407-8912-4bf3-9cf6-bd5b111c64f0 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:12.688760 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64720407-8912-4bf3-9cf6-bd5b111c64f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:12.689031 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64720407-8912-4bf3-9cf6-bd5b111c64f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:12.695779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64720407-8912-4bf3-9cf6-bd5b111c64f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:12.697223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64720407-8912-4bf3-9cf6-bd5b111c64f0 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:12.702129 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35547478-a64e-4ff1-95cc-279fc5d6cb1b None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:12.708498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-64720407-8912-4bf3-9cf6-bd5b111c64f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:12.708498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:12.708498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:12.708498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:12.708498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:12.708498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:12.708498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:12.708498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:12.708498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:12.708498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:12.708498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:12.708498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:12.708498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:00:12.708498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:00:12.708498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:00:12.709569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:00:12.709569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:12.709569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:12.709569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:12.709569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:12.709569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:12.709569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:12.709569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:12.709569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:12.709569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:12.709569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:12.709569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:12.709569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:12.709569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:12.709569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:12.709569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:12.709569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:12.710590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:00:12.710590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:00:12.710590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:00:12.710590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:00:12.710590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:00:12.710590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:00:12.710590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:12.710590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:12.710590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 389/805] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:12.712139 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64720407-8912-4bf3-9cf6-bd5b111c64f0 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:12.723855 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:12.737974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23996d31-be38-474b-8fc6-d7c840039ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:12.738589 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23996d31-be38-474b-8fc6-d7c840039ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:12.738854 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23996d31-be38-474b-8fc6-d7c840039ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:12.739125 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23996d31-be38-474b-8fc6-d7c840039ea9 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:12.739939 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23996d31-be38-474b-8fc6-d7c840039ea9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:12.740241 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23996d31-be38-474b-8fc6-d7c840039ea9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:12.741660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23996d31-be38-474b-8fc6-d7c840039ea9 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:12.755686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:12.758962 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 417/806] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Aug 23 07:00:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:12.759880 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23996d31-be38-474b-8fc6-d7c840039ea9 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:12.818957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64720407-8912-4bf3-9cf6-bd5b111c64f0 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:12.858087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23996d31-be38-474b-8fc6-d7c840039ea9 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:12.860375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d321b36-a7e3-4a9b-b545-f1cc81485803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:12.861029 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d321b36-a7e3-4a9b-b545-f1cc81485803 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:12.861290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d321b36-a7e3-4a9b-b545-f1cc81485803 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:12.861558 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d321b36-a7e3-4a9b-b545-f1cc81485803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:12.872357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d321b36-a7e3-4a9b-b545-f1cc81485803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:12.873882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d321b36-a7e3-4a9b-b545-f1cc81485803 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:12.888396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-5d321b36-a7e3-4a9b-b545-f1cc81485803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:12.888396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:12.888396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:12.888396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:12.888396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:12.888396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:12.888396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:12.888396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:12.888396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:12.888396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:12.888396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:12.888396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:12.888396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:12.888396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:12.888396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:12.888396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:12.889450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:12.889450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:12.889450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:12.889450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:12.889450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:12.889450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:12.889450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:12.889450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:12.889450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:12.889450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:12.889450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:12.889450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:12.889450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:12.889450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:12.889450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:12.889450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:12.889450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:12.890330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:12.890330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:12.890330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:12.890330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 390/807] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:12.890751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf95d83-ce19-4e5c-9424-153ff4ab9176 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:12.891004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf95d83-ce19-4e5c-9424-153ff4ab9176 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:12.891280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf95d83-ce19-4e5c-9424-153ff4ab9176 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:12.892512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 391/808] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:12.893189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf95d83-ce19-4e5c-9424-153ff4ab9176 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:12.896867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad81346-8540-4e2d-b458-97710f80e6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:12.897893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad81346-8540-4e2d-b458-97710f80e6a8 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:12.899046 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad81346-8540-4e2d-b458-97710f80e6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:12.899327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad81346-8540-4e2d-b458-97710f80e6a8 None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:12.900022 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad81346-8540-4e2d-b458-97710f80e6a8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:12.900331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad81346-8540-4e2d-b458-97710f80e6a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:12.901726 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad81346-8540-4e2d-b458-97710f80e6a8 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:12.906646 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 418/809] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Aug 23 07:00:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:12.907459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad81346-8540-4e2d-b458-97710f80e6a8 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:12.952507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:12.997745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:13.006199 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf95d83-ce19-4e5c-9424-153ff4ab9176 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:13.007435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad81346-8540-4e2d-b458-97710f80e6a8 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:13.039359 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbc7a5-ed1c-49e1-9087-8069cb46b0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:13.039994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbc7a5-ed1c-49e1-9087-8069cb46b0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:13.040273 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbc7a5-ed1c-49e1-9087-8069cb46b0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:13.040546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbc7a5-ed1c-49e1-9087-8069cb46b0f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:13.041042 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d466c8-069e-4fd1-93d8-747b10eb5736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:13.041268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbc7a5-ed1c-49e1-9087-8069cb46b0f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:13.041573 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbc7a5-ed1c-49e1-9087-8069cb46b0f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:13.041661 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d466c8-069e-4fd1-93d8-747b10eb5736 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:13.041926 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d466c8-069e-4fd1-93d8-747b10eb5736 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:13.042196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d466c8-069e-4fd1-93d8-747b10eb5736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:13.043018 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbc7a5-ed1c-49e1-9087-8069cb46b0f5 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:13.052206 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d466c8-069e-4fd1-93d8-747b10eb5736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:13.053639 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d466c8-069e-4fd1-93d8-747b10eb5736 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:13.054947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 392/810] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:00:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:13.055788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbc7a5-ed1c-49e1-9087-8069cb46b0f5 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:13.069683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-36d466c8-069e-4fd1-93d8-747b10eb5736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:13.069683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:13.069683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:13.069683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:13.069683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:13.069683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:13.069683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:13.069683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:13.069683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:13.069683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:13.069683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:13.069683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:13.069683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:13.069683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:13.069683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:13.069683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:13.070634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:13.070634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:13.070634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:13.070634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:13.070634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:13.070634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:13.070634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:13.070634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:13.070634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:13.070634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:13.070634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:13.070634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:13.070634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:13.070634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:13.070634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:13.070634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:13.070634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:13.076302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:13.076302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:13.076302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:13.076302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 419/811] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:13.076302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f78a0d-4a77-4935-ad78-cfa1c7df186a None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:13.076302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f78a0d-4a77-4935-ad78-cfa1c7df186a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:13.076302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f78a0d-4a77-4935-ad78-cfa1c7df186a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:13.090432 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-22f78a0d-4a77-4935-ad78-cfa1c7df186a None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:13.091672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-22f78a0d-4a77-4935-ad78-cfa1c7df186a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:13.155337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbc7a5-ed1c-49e1-9087-8069cb46b0f5 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:13.156882 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 420/812] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:13.157688 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f78a0d-4a77-4935-ad78-cfa1c7df186a None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:13.185395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:13.193493 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e0129-9bca-4196-92e6-0ff9732ea339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:13.194104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e0129-9bca-4196-92e6-0ff9732ea339 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:13.194382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e0129-9bca-4196-92e6-0ff9732ea339 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:13.194653 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e0129-9bca-4196-92e6-0ff9732ea339 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:13.195325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e0129-9bca-4196-92e6-0ff9732ea339 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:13.195653 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e0129-9bca-4196-92e6-0ff9732ea339 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:13.197034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e0129-9bca-4196-92e6-0ff9732ea339 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:13.209361 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 393/813] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:13.210156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e0129-9bca-4196-92e6-0ff9732ea339 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:13.225522 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:13.270088 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f78a0d-4a77-4935-ad78-cfa1c7df186a None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:13.305099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463f29d-2183-4dba-8648-9d0b16b74192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:13.305700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463f29d-2183-4dba-8648-9d0b16b74192 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:13.305972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463f29d-2183-4dba-8648-9d0b16b74192 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:13.306238 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463f29d-2183-4dba-8648-9d0b16b74192 None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:13.306858 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463f29d-2183-4dba-8648-9d0b16b74192 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:13.307153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463f29d-2183-4dba-8648-9d0b16b74192 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:13.308545 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463f29d-2183-4dba-8648-9d0b16b74192 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:13.314341 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 421/814] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Aug 23 07:00:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:13.315107 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463f29d-2183-4dba-8648-9d0b16b74192 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:13.324452 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e0129-9bca-4196-92e6-0ff9732ea339 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:13.359424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b68e8-7baa-4ebe-a92b-a44abba464d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:13.360296 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b68e8-7baa-4ebe-a92b-a44abba464d8 None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:13.360463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b68e8-7baa-4ebe-a92b-a44abba464d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:13.360641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b68e8-7baa-4ebe-a92b-a44abba464d8 None admin] PATH_INFO: `/v3/projects/351a6777525c4207826d7df0d3768e99/users/fd88ec8b477443af95bff9af03471adf/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:13.362512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b68e8-7baa-4ebe-a92b-a44abba464d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=351a6777525c4207826d7df0d3768e99, user_id=fd88ec8b477443af95bff9af03471adf, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:13.364010 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b68e8-7baa-4ebe-a92b-a44abba464d8 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:13.377904 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 394/815] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 07:00:13 2022] PUT /identity/v3/projects/351a6777525c4207826d7df0d3768e99/users/fd88ec8b477443af95bff9af03471adf/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:00:13.392432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b68e8-7baa-4ebe-a92b-a44abba464d8 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:13.413208 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:13.427259 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463f29d-2183-4dba-8648-9d0b16b74192 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:13.465426 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1611e01-7750-4146-8d3c-8c96c770381a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:13.466047 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1611e01-7750-4146-8d3c-8c96c770381a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:13.466326 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1611e01-7750-4146-8d3c-8c96c770381a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:13.466592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1611e01-7750-4146-8d3c-8c96c770381a None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:13.467246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1611e01-7750-4146-8d3c-8c96c770381a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:13.467573 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1611e01-7750-4146-8d3c-8c96c770381a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:13.468962 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1611e01-7750-4146-8d3c-8c96c770381a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:13.482087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 422/816] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:13.482928 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1611e01-7750-4146-8d3c-8c96c770381a None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:13.498288 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:13.534909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b68e8-7baa-4ebe-a92b-a44abba464d8 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:13.574914 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9cc21-80bb-494b-b803-53534a5f13d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:13.575599 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9cc21-80bb-494b-b803-53534a5f13d2 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:13.575880 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9cc21-80bb-494b-b803-53534a5f13d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:13.576153 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9cc21-80bb-494b-b803-53534a5f13d2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:13.585447 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9cc21-80bb-494b-b803-53534a5f13d2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:13.586906 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9cc21-80bb-494b-b803-53534a5f13d2 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:13.597652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1611e01-7750-4146-8d3c-8c96c770381a None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:13.598229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-36f9cc21-80bb-494b-b803-53534a5f13d2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:00:13.598229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:13.598229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:13.598229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:13.598229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:13.598229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:13.598229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:13.598229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:13.598229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:13.598229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:13.598229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:13.598229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:13.598229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:00:13.598229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:00:13.598229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:00:13.598229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:00:13.599169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:13.599169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:13.599169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:00:13.599169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:13.599169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:00:13.599169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:13.599169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:13.599169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:13.599169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:13.599169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:13.599169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:13.599169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:13.599169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:13.599169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:13.599169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:13.599169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:13.599169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:13.599169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:13.600375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:13.600375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:13.600375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:00:13.600375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:00:13.600375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:00:13.600375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:00:13.600375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:00:13.600375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:00:13.600375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:00:13.600375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:13.600375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 395/817] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Aug 23 07:00:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:13.600375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9cc21-80bb-494b-b803-53534a5f13d2 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:13.642165 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806b6ee-7a78-4e50-bf80-ce0021d0a014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:13.643119 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806b6ee-7a78-4e50-bf80-ce0021d0a014 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:13.643567 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806b6ee-7a78-4e50-bf80-ce0021d0a014 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:13.643973 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806b6ee-7a78-4e50-bf80-ce0021d0a014 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:13.644763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806b6ee-7a78-4e50-bf80-ce0021d0a014 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:13.646527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806b6ee-7a78-4e50-bf80-ce0021d0a014 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:13.672622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 423/818] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:00:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:00:13.673395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806b6ee-7a78-4e50-bf80-ce0021d0a014 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:13.678145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:13.736456 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9cc21-80bb-494b-b803-53534a5f13d2 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:13.778716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:13.785807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb124a89-0b20-4dc8-801a-e3bb92dfe349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:13.787213 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb124a89-0b20-4dc8-801a-e3bb92dfe349 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:13.787540 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb124a89-0b20-4dc8-801a-e3bb92dfe349 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:13.788200 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb124a89-0b20-4dc8-801a-e3bb92dfe349 None admin] PATH_INFO: `/v3/services/network` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:13.788946 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb124a89-0b20-4dc8-801a-e3bb92dfe349 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:13.790495 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb124a89-0b20-4dc8-801a-e3bb92dfe349 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:13.800650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-eb124a89-0b20-4dc8-801a-e3bb92dfe349 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:00:13.800650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:13.800650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:13.800650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:13.800650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:13.800650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:13.800650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:13.800650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:13.800650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:00:13.800650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:13.800650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:13.800650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:13.800650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:13.800650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:13.800650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:13.800650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:13.800650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:13.802320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:13.802320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:13.802320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:13.802320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:13.802320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:13.802320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:13.802320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:13.802320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:13.802320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:13.802320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:13.802320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:13.802320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:13.802320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:13.802320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:13.802320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:13.802320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:13.803445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:13.803445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:13.803445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:13.803445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:13.803445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:13.803445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:13.803445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:13.803445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:13.803445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:13.803445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:13.803445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:13.803445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:13.803445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:00:13.803445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:13.805257 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 396/819] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:13 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:13.806286 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb124a89-0b20-4dc8-801a-e3bb92dfe349 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:13.822280 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806b6ee-7a78-4e50-bf80-ce0021d0a014 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:13.875508 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42814fc-76a3-4de5-8017-6acb999ee415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:13.876149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42814fc-76a3-4de5-8017-6acb999ee415 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:13.876425 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42814fc-76a3-4de5-8017-6acb999ee415 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:13.876696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42814fc-76a3-4de5-8017-6acb999ee415 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:13.885125 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42814fc-76a3-4de5-8017-6acb999ee415 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:13.886590 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42814fc-76a3-4de5-8017-6acb999ee415 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:13.900289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-b42814fc-76a3-4de5-8017-6acb999ee415 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:13.900289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:13.900289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:13.900289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:13.900289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:13.900289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:13.900289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:13.900289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:13.900289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:13.900289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:13.900289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:13.900289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:13.900289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:00:13.900289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:00:13.900289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:00:13.901226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:00:13.901226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:13.901226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:13.901226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:13.901226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:13.901226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:13.901226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:13.901226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:13.901226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:13.901226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:13.901226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:13.901226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:13.901226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:13.901226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:13.901226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:13.901226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:13.901226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:13.902602 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:00:13.902602 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:00:13.902602 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:00:13.902602 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:00:13.902602 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:00:13.902602 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:00:13.902602 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:13.902602 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:13.902602 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 424/820] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:13.902602 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42814fc-76a3-4de5-8017-6acb999ee415 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:13.928242 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb124a89-0b20-4dc8-801a-e3bb92dfe349 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:13.975744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:13.982261 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d90204-2b16-4114-a2ed-1d33aeaa6ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:13.983433 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d90204-2b16-4114-a2ed-1d33aeaa6ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:13.983961 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d90204-2b16-4114-a2ed-1d33aeaa6ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:13.984597 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d90204-2b16-4114-a2ed-1d33aeaa6ab8 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:13.986428 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d90204-2b16-4114-a2ed-1d33aeaa6ab8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:13.986888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d90204-2b16-4114-a2ed-1d33aeaa6ab8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:13.988528 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d90204-2b16-4114-a2ed-1d33aeaa6ab8 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:14.015462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 397/821] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Aug 23 07:00:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:14.020653 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d90204-2b16-4114-a2ed-1d33aeaa6ab8 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:14.025227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42814fc-76a3-4de5-8017-6acb999ee415 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:14.075938 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4c14f-c39a-49bb-884b-b3d8af7f9043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:14.077269 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4c14f-c39a-49bb-884b-b3d8af7f9043 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:14.077727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4c14f-c39a-49bb-884b-b3d8af7f9043 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:14.078172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4c14f-c39a-49bb-884b-b3d8af7f9043 None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:14.079291 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4c14f-c39a-49bb-884b-b3d8af7f9043 None admin] RBAC: Adding query filter params (name=network) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:14.080517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:14.081055 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4c14f-c39a-49bb-884b-b3d8af7f9043 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:14.084331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4c14f-c39a-49bb-884b-b3d8af7f9043 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:14.092721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 425/822] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 23 07:00:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:14.093996 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4c14f-c39a-49bb-884b-b3d8af7f9043 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:14.134826 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d90204-2b16-4114-a2ed-1d33aeaa6ab8 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:14.172510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c1fa6-7b35-45ca-b2c5-534430cf583f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:14.173301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c1fa6-7b35-45ca-b2c5-534430cf583f None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:14.173673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c1fa6-7b35-45ca-b2c5-534430cf583f None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:14.174189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c1fa6-7b35-45ca-b2c5-534430cf583f None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:14.175342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c1fa6-7b35-45ca-b2c5-534430cf583f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:14.175770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c1fa6-7b35-45ca-b2c5-534430cf583f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:14.177837 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c1fa6-7b35-45ca-b2c5-534430cf583f None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:14.195523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 398/823] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:00:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:14.195523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c1fa6-7b35-45ca-b2c5-534430cf583f None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:14.231035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4c14f-c39a-49bb-884b-b3d8af7f9043 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:14.264369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:14.275636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5415f5b-fae6-4f59-9474-a6961479b960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:14.276764 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5415f5b-fae6-4f59-9474-a6961479b960 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:14.277191 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5415f5b-fae6-4f59-9474-a6961479b960 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:14.277746 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5415f5b-fae6-4f59-9474-a6961479b960 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:14.316117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c1fa6-7b35-45ca-b2c5-534430cf583f None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:14.331648 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:14.357995 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e00968-13eb-4b9b-a8a8-451e4ffe209f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:14.358801 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e00968-13eb-4b9b-a8a8-451e4ffe209f None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:14.359174 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e00968-13eb-4b9b-a8a8-451e4ffe209f None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:14.359571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e00968-13eb-4b9b-a8a8-451e4ffe209f None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:14.360304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e00968-13eb-4b9b-a8a8-451e4ffe209f None admin] RBAC: Adding query filter params (type=network) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:14.360702 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e00968-13eb-4b9b-a8a8-451e4ffe209f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:14.362191 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e00968-13eb-4b9b-a8a8-451e4ffe209f None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:14.367946 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 399/824] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 23 07:00:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:14.369110 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e00968-13eb-4b9b-a8a8-451e4ffe209f None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:14.374802 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5415f5b-fae6-4f59-9474-a6961479b960 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:14.376548 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5415f5b-fae6-4f59-9474-a6961479b960 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:14.487220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e00968-13eb-4b9b-a8a8-451e4ffe209f None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:14.512824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 426/825] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:00:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:00:14.512824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5415f5b-fae6-4f59-9474-a6961479b960 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:14.530649 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db474d47-659d-4fc9-ad3b-b5b1274c6e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:14.531719 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db474d47-659d-4fc9-ad3b-b5b1274c6e40 None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:14.532094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db474d47-659d-4fc9-ad3b-b5b1274c6e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:14.532458 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db474d47-659d-4fc9-ad3b-b5b1274c6e40 None admin] PATH_INFO: `/v3/projects/351a6777525c4207826d7df0d3768e99/users/b3f1ae88e29f4172a7db7393d517a893/roles/2d62b8b8395745c7983c6c8d100d985b` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:14.543246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:14.554861 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db474d47-659d-4fc9-ad3b-b5b1274c6e40 None admin] RBAC: Authorizing `identity:create_grant(project_id=351a6777525c4207826d7df0d3768e99, user_id=b3f1ae88e29f4172a7db7393d517a893, role_id=2d62b8b8395745c7983c6c8d100d985b)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:14.556671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db474d47-659d-4fc9-ad3b-b5b1274c6e40 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:14.579445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 400/826] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 07:00:14 2022] PUT /identity/v3/projects/351a6777525c4207826d7df0d3768e99/users/b3f1ae88e29f4172a7db7393d517a893/roles/2d62b8b8395745c7983c6c8d100d985b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:00:14.579445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db474d47-659d-4fc9-ad3b-b5b1274c6e40 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:14.616164 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:14.681798 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5415f5b-fae6-4f59-9474-a6961479b960 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:14.714660 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db474d47-659d-4fc9-ad3b-b5b1274c6e40 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:14.757979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61572b00-0da7-4041-acaa-800c4b265ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:14.758769 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61572b00-0da7-4041-acaa-800c4b265ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:14.759676 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61572b00-0da7-4041-acaa-800c4b265ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:14.760108 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61572b00-0da7-4041-acaa-800c4b265ba1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:14.786712 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823e94a-1caa-4e50-8f54-244618c94a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:14.787999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61572b00-0da7-4041-acaa-800c4b265ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:14.789488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61572b00-0da7-4041-acaa-800c4b265ba1 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:14.790431 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823e94a-1caa-4e50-8f54-244618c94a93 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:14.790713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823e94a-1caa-4e50-8f54-244618c94a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:14.790981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823e94a-1caa-4e50-8f54-244618c94a93 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:14.791698 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823e94a-1caa-4e50-8f54-244618c94a93 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:14.793072 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823e94a-1caa-4e50-8f54-244618c94a93 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:14.814146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-61572b00-0da7-4041-acaa-800c4b265ba1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:14.814146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:14.814146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:14.814146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:14.814146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:14.814146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:14.814146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:14.814146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:14.814146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:14.814146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:14.814146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:14.814146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:14.814146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:14.814146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:14.814146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:14.814146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:14.815234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:14.815234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:14.815234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:14.815234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:14.815234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:14.815234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:14.815234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:14.815234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:14.815234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:14.815234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:14.815234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:14.815234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:14.815234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:14.815234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:14.815234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:14.815234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:14.815234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:14.816331 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:14.816331 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:14.816331 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:14.818871 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 401/827] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:14.825295 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61572b00-0da7-4041-acaa-800c4b265ba1 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:14.831446 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 427/828] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:00:14 2022] POST /identity/v3/endpoints => generated 337 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:00:14.941327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:14.977572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61572b00-0da7-4041-acaa-800c4b265ba1 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:15.026464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004a15a-2896-4232-a6b9-6505d2e063b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:15.026991 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004a15a-2896-4232-a6b9-6505d2e063b8 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:15.027268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004a15a-2896-4232-a6b9-6505d2e063b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:15.027575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004a15a-2896-4232-a6b9-6505d2e063b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:15.028220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004a15a-2896-4232-a6b9-6505d2e063b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:15.028523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004a15a-2896-4232-a6b9-6505d2e063b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:15.029904 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004a15a-2896-4232-a6b9-6505d2e063b8 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:15.044110 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 402/829] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:15.049759 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823e94a-1caa-4e50-8f54-244618c94a93 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:15.157672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:15.178382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823e94a-1caa-4e50-8f54-244618c94a93 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:15.219369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe1794-b773-467e-9403-7b167474ddf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:15.220610 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe1794-b773-467e-9403-7b167474ddf7 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:15.221339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe1794-b773-467e-9403-7b167474ddf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:15.221834 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe1794-b773-467e-9403-7b167474ddf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:15.236528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe1794-b773-467e-9403-7b167474ddf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:15.238196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe1794-b773-467e-9403-7b167474ddf7 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:15.253091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-0bfe1794-b773-467e-9403-7b167474ddf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:15.253091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:15.253091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:15.253091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:15.253091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:15.253091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:15.253091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:15.253091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:15.253091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:15.253091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:15.253091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:15.253091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:15.253091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:00:15.253091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:00:15.253091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:00:15.254281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:00:15.254281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:15.254281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:15.254281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:15.254281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:15.254281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:15.254281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:15.254281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:15.254281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:15.254281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:15.254281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:15.254281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:15.254281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:15.254281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:15.254281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:15.254281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:15.254281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:15.255440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:00:15.255440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:00:15.255440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:00:15.255440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:00:15.255440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:00:15.255440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:00:15.255440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:15.255440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:15.255440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 428/830] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:15.260289 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004a15a-2896-4232-a6b9-6505d2e063b8 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:15.372828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004a15a-2896-4232-a6b9-6505d2e063b8 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:15.407601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62f70c-d888-48d1-87f9-a77991165268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:15.408369 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62f70c-d888-48d1-87f9-a77991165268 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:15.408742 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62f70c-d888-48d1-87f9-a77991165268 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:15.409107 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62f70c-d888-48d1-87f9-a77991165268 None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:15.409918 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62f70c-d888-48d1-87f9-a77991165268 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:15.410324 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62f70c-d888-48d1-87f9-a77991165268 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:15.411945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62f70c-d888-48d1-87f9-a77991165268 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:15.423755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:15.424104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 403/831] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:00:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:15.432503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe1794-b773-467e-9403-7b167474ddf7 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:15.472447 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a64156-254e-44ee-9a50-774473818d5a None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:15.472733 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a64156-254e-44ee-9a50-774473818d5a None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:15.473009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a64156-254e-44ee-9a50-774473818d5a None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:15.474652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 404/832] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:15.480051 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076e23f-c1f7-4889-ad67-3e66f2136f4a None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:15.480332 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076e23f-c1f7-4889-ad67-3e66f2136f4a None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:15.480606 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076e23f-c1f7-4889-ad67-3e66f2136f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:15.504945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-b076e23f-c1f7-4889-ad67-3e66f2136f4a None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:15.506282 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b076e23f-c1f7-4889-ad67-3e66f2136f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:15.563595 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe1794-b773-467e-9403-7b167474ddf7 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:15.583444 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 405/833] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:15.608544 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ee8ba-fcf7-48ac-878f-143c6d916486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:15.609158 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ee8ba-fcf7-48ac-878f-143c6d916486 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:15.609433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ee8ba-fcf7-48ac-878f-143c6d916486 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:15.609702 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ee8ba-fcf7-48ac-878f-143c6d916486 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:15.610442 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ee8ba-fcf7-48ac-878f-143c6d916486 None admin] RBAC: Adding query filter params (role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=a8d2cd93042c4583b63afe629d47fce0) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:15.610744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ee8ba-fcf7-48ac-878f-143c6d916486 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:15.612230 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ee8ba-fcf7-48ac-878f-143c6d916486 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:15.618675 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 429/834] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:00:15 2022] GET /identity/v3/role_assignments?user.id=a8d2cd93042c4583b63afe629d47fce0&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:15.684561 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119d649-372a-41e4-9d23-33214abd13ca None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:15.684920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119d649-372a-41e4-9d23-33214abd13ca None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:15.685204 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119d649-372a-41e4-9d23-33214abd13ca None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:15.686774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 406/835] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:15.692517 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86b232-06ae-4f46-bf63-daa3bb6dd132 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:15.693107 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86b232-06ae-4f46-bf63-daa3bb6dd132 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:15.693185 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86b232-06ae-4f46-bf63-daa3bb6dd132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:15.713406 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-ec86b232-06ae-4f46-bf63-daa3bb6dd132 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:15.714625 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ec86b232-06ae-4f46-bf63-daa3bb6dd132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:15.716814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:15.788828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 430/836] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:15.856482 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119d649-372a-41e4-9d23-33214abd13ca None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:15.967758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119d649-372a-41e4-9d23-33214abd13ca None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:15.992988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:16.004862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f8411-b34f-478a-81d8-658d2ea17a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:16.005995 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f8411-b34f-478a-81d8-658d2ea17a72 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:16.006478 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f8411-b34f-478a-81d8-658d2ea17a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:16.006952 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f8411-b34f-478a-81d8-658d2ea17a72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:16.017931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f8411-b34f-478a-81d8-658d2ea17a72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:00:16.023086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-eb5f8411-b34f-478a-81d8-658d2ea17a72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:00:16.023086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:16.023086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:16.023086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:16.023086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:16.023086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:16.023086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:16.023086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:16.023086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:16.023086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:16.023086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:16.023086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:16.023086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:00:16.023086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:00:16.023086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:00:16.023086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:00:16.024082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:00:16.024082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:00:16.024082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:00:16.024082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:00:16.024082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:16.024082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:16.024082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:16.024082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:16.024082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:16.024082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:16.024082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:16.024082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:16.024082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:16.024082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:16.024082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:16.024082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:16.024942 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:16.024942 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:16.024942 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:16.024942 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:16.024942 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:00:16.024942 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:00:16.024942 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:00:16.024942 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:00:16.024942 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:00:16.024942 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:00:16.024942 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:00:16.024942 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:16.027696 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 407/837] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:00:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:16.035459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86b232-06ae-4f46-bf63-daa3bb6dd132 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:16.089891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f8411-b34f-478a-81d8-658d2ea17a72 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:16.141463 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86b232-06ae-4f46-bf63-daa3bb6dd132 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:16.189181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae52ec5-63c1-45ef-bf18-b506dbcae938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:16.189800 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae52ec5-63c1-45ef-bf18-b506dbcae938 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:16.190074 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae52ec5-63c1-45ef-bf18-b506dbcae938 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:16.190862 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae52ec5-63c1-45ef-bf18-b506dbcae938 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:16.191558 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae52ec5-63c1-45ef-bf18-b506dbcae938 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:16.192137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae52ec5-63c1-45ef-bf18-b506dbcae938 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:16.193889 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae52ec5-63c1-45ef-bf18-b506dbcae938 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:16.201734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:16.202985 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f8411-b34f-478a-81d8-658d2ea17a72 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:16.209538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 431/838] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 07:00:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:16.215541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae52ec5-63c1-45ef-bf18-b506dbcae938 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:16.245161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9692b-8205-44fb-8300-92ef32e8159f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:16.246184 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9692b-8205-44fb-8300-92ef32e8159f None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:16.246303 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9692b-8205-44fb-8300-92ef32e8159f None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:16.246501 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9692b-8205-44fb-8300-92ef32e8159f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:16.247136 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9692b-8205-44fb-8300-92ef32e8159f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:16.248702 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9692b-8205-44fb-8300-92ef32e8159f None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:16.258026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-3af9692b-8205-44fb-8300-92ef32e8159f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:00:16.258026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:16.258026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:16.258026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:16.258026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:16.258026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:16.258026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:16.258026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:16.258026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:00:16.258026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:16.258026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:16.258026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:16.258026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:16.258026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:16.258026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:16.258026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:16.258026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:16.259201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:16.259201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:16.259201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:16.259201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:16.259201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:16.259201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:16.259201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:16.259201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:16.259201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:16.259201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:16.259201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:16.259201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:16.259201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:16.259201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:16.259201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:16.259201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:16.260059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:16.260059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:16.260059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:16.260059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:16.260059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:16.260059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:16.260059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:16.260059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:16.260059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:16.260059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:16.260059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:16.260059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:16.260059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:00:16.260059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:16.260059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 408/839] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 07:00:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:16.267299 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9692b-8205-44fb-8300-92ef32e8159f None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:16.320971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:16.335366 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae52ec5-63c1-45ef-bf18-b506dbcae938 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:16.370182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65706a-d542-4f53-b57e-7825ad171028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:16.370870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65706a-d542-4f53-b57e-7825ad171028 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:16.371091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65706a-d542-4f53-b57e-7825ad171028 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:16.371366 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65706a-d542-4f53-b57e-7825ad171028 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:16.379990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9692b-8205-44fb-8300-92ef32e8159f None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:16.382589 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65706a-d542-4f53-b57e-7825ad171028 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:16.384106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65706a-d542-4f53-b57e-7825ad171028 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:16.399007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-ee65706a-d542-4f53-b57e-7825ad171028 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:16.399007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:16.399007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:16.399007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:16.399007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:16.399007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:16.399007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:16.399007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:16.399007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:16.399007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:16.399007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:16.399007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:16.399007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:16.399007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:16.399007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:16.399007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:16.400084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:16.400084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:16.400084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:16.400084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:16.400084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:16.400084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:16.400084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:16.400084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:16.400084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:16.400084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:16.400084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:16.400084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:16.400084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:16.400084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:16.400084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:16.400084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:16.400084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:16.401225 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:16.401225 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:16.401225 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:16.406170 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 432/840] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:16.407081 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65706a-d542-4f53-b57e-7825ad171028 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:16.420662 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544eb71-33e0-4bae-bed5-1d24ccad6a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:16.421504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544eb71-33e0-4bae-bed5-1d24ccad6a04 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:16.421919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544eb71-33e0-4bae-bed5-1d24ccad6a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:16.422327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544eb71-33e0-4bae-bed5-1d24ccad6a04 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:16.423205 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544eb71-33e0-4bae-bed5-1d24ccad6a04 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:16.423680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544eb71-33e0-4bae-bed5-1d24ccad6a04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:16.425263 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544eb71-33e0-4bae-bed5-1d24ccad6a04 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:16.430584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 409/841] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 07:00:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:16.435751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544eb71-33e0-4bae-bed5-1d24ccad6a04 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:16.467745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:16.515619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65706a-d542-4f53-b57e-7825ad171028 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:16.547641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544eb71-33e0-4bae-bed5-1d24ccad6a04 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:16.550516 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc413502-5661-43aa-b905-5eabf4d5e2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:16.551546 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc413502-5661-43aa-b905-5eabf4d5e2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:16.551546 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc413502-5661-43aa-b905-5eabf4d5e2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:16.551804 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc413502-5661-43aa-b905-5eabf4d5e2c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:16.552732 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc413502-5661-43aa-b905-5eabf4d5e2c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:16.553099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc413502-5661-43aa-b905-5eabf4d5e2c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:16.554558 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc413502-5661-43aa-b905-5eabf4d5e2c6 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:16.563653 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:16.571778 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 433/842] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:16.573740 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc413502-5661-43aa-b905-5eabf4d5e2c6 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:16.589314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc338e-4eac-4727-892f-565b6e87df1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:16.590261 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc338e-4eac-4727-892f-565b6e87df1b None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:16.590757 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc338e-4eac-4727-892f-565b6e87df1b None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:16.591255 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc338e-4eac-4727-892f-565b6e87df1b None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:16.592249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc338e-4eac-4727-892f-565b6e87df1b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:16.592743 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc338e-4eac-4727-892f-565b6e87df1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:16.594383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc338e-4eac-4727-892f-565b6e87df1b None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:16.600440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 410/843] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 07:00:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:16.784165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb90aa8-c998-4860-a0c6-e36d9e1fa261 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:16.784359 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb90aa8-c998-4860-a0c6-e36d9e1fa261 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:16.784629 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb90aa8-c998-4860-a0c6-e36d9e1fa261 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:16.786195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 411/844] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:16.791342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c84a16-7d07-4429-8818-8146862c6f3d None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:16.791919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c84a16-7d07-4429-8818-8146862c6f3d None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:16.792391 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c84a16-7d07-4429-8818-8146862c6f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:16.813851 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-f0c84a16-7d07-4429-8818-8146862c6f3d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:16.815283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f0c84a16-7d07-4429-8818-8146862c6f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:16.823227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc413502-5661-43aa-b905-5eabf4d5e2c6 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:16.839637 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:16.869564 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216b2ef-2c20-45f7-b99d-ad93251ddc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:16.870337 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216b2ef-2c20-45f7-b99d-ad93251ddc6f None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:16.870662 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216b2ef-2c20-45f7-b99d-ad93251ddc6f None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:16.870932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216b2ef-2c20-45f7-b99d-ad93251ddc6f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:16.881621 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216b2ef-2c20-45f7-b99d-ad93251ddc6f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:16.883840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216b2ef-2c20-45f7-b99d-ad93251ddc6f None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:16.895873 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 412/845] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:16.896257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-2216b2ef-2c20-45f7-b99d-ad93251ddc6f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:00:16.896257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:16.896257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:16.896257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:16.896257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:16.896257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:16.896257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:16.896257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:16.896257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:16.896257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:16.896257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:16.896257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:16.896257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:00:16.896257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:00:16.896257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:00:16.896257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:00:16.897176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:16.897176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:16.897176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:00:16.897176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:16.897176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:00:16.897176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:16.897176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:16.897176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:16.897176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:16.897176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:16.897176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:16.897176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:16.897176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:16.897176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:16.897176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:16.897176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:16.897176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:16.897176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:16.898106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:16.898106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:16.898106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:00:16.898106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:00:16.898106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:00:16.898106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:00:16.898106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:00:16.898106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:00:16.898106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:00:16.898106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:16.898106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 434/846] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:00:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:16.903070 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c84a16-7d07-4429-8818-8146862c6f3d None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:17.009958 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c84a16-7d07-4429-8818-8146862c6f3d None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:17.019085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:17.049048 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827696fc-c2c3-4e4a-b3e8-94f5b1ff5ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:17.049585 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827696fc-c2c3-4e4a-b3e8-94f5b1ff5ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:17.049859 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827696fc-c2c3-4e4a-b3e8-94f5b1ff5ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:17.050129 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827696fc-c2c3-4e4a-b3e8-94f5b1ff5ea3 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:17.050868 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827696fc-c2c3-4e4a-b3e8-94f5b1ff5ea3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:17.051169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827696fc-c2c3-4e4a-b3e8-94f5b1ff5ea3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:17.052928 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827696fc-c2c3-4e4a-b3e8-94f5b1ff5ea3 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:17.071204 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 413/847] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Aug 23 07:00:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:17.076645 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216b2ef-2c20-45f7-b99d-ad93251ddc6f None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:17.175091 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827696fc-c2c3-4e4a-b3e8-94f5b1ff5ea3 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:17.181347 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216b2ef-2c20-45f7-b99d-ad93251ddc6f None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:17.216902 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7fa31-feae-4d0e-9dd4-534270731d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:17.217515 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7fa31-feae-4d0e-9dd4-534270731d08 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:17.217795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7fa31-feae-4d0e-9dd4-534270731d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:17.218071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7fa31-feae-4d0e-9dd4-534270731d08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:17.259829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:17.288503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a7fa31-feae-4d0e-9dd4-534270731d08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:17.289981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a7fa31-feae-4d0e-9dd4-534270731d08 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:17.295711 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827696fc-c2c3-4e4a-b3e8-94f5b1ff5ea3 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:17.332718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e7b7b-4b28-4461-8a18-d5c6f011fe3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:17.333422 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e7b7b-4b28-4461-8a18-d5c6f011fe3a None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:17.333699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e7b7b-4b28-4461-8a18-d5c6f011fe3a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:17.333968 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e7b7b-4b28-4461-8a18-d5c6f011fe3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:17.345086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e7b7b-4b28-4461-8a18-d5c6f011fe3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:00:17.350912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-880e7b7b-4b28-4461-8a18-d5c6f011fe3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:00:17.350912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:17.350912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:17.350912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:17.350912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:17.350912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:17.350912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:17.350912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:17.350912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:17.350912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:17.350912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:17.350912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:17.350912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:00:17.350912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:00:17.350912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:00:17.350912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:00:17.353081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:00:17.353081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:00:17.353081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:00:17.353081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:00:17.353081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:17.353081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:17.353081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:17.353081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:17.353081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:17.353081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:17.353081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:17.353081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:17.353081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:17.353081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:17.353081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:17.353081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:17.355078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:17.355078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:17.355078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:17.355078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:17.355078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:00:17.355078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:00:17.355078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:00:17.355078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:00:17.355078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:00:17.355078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:00:17.355078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:00:17.355078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:17.363460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 414/848] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 07:00:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:17.363460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e7b7b-4b28-4461-8a18-d5c6f011fe3a None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:17.385788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:17.397245 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 435/849] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:00:17 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:00:17.398583 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7fa31-feae-4d0e-9dd4-534270731d08 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:17.468000 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:17.470454 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e7b7b-4b28-4461-8a18-d5c6f011fe3a None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:17.498995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7fa31-feae-4d0e-9dd4-534270731d08 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:17.509078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3f154-a445-4490-96c8-dffff205c73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:17.510035 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3f154-a445-4490-96c8-dffff205c73a None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:17.510485 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3f154-a445-4490-96c8-dffff205c73a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:17.510948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3f154-a445-4490-96c8-dffff205c73a None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:17.511844 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3f154-a445-4490-96c8-dffff205c73a None admin] RBAC: Adding query filter params (name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:17.512316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3f154-a445-4490-96c8-dffff205c73a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:17.514001 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3f154-a445-4490-96c8-dffff205c73a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:17.527679 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 415/850] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 07:00:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:17.529498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3f154-a445-4490-96c8-dffff205c73a None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:17.534206 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995b457-2a2c-49c9-b6f0-ff2ca9970c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:17.534561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995b457-2a2c-49c9-b6f0-ff2ca9970c6c None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:17.534846 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995b457-2a2c-49c9-b6f0-ff2ca9970c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:17.535118 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995b457-2a2c-49c9-b6f0-ff2ca9970c6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:17.546223 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995b457-2a2c-49c9-b6f0-ff2ca9970c6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:17.547712 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995b457-2a2c-49c9-b6f0-ff2ca9970c6c None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:17.562449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-3995b457-2a2c-49c9-b6f0-ff2ca9970c6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:17.562449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:17.562449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:17.562449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:17.562449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:17.562449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:17.562449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:17.562449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:17.562449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:17.562449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:17.562449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:17.562449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:17.562449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:17.562449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:17.562449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:17.562449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:17.563450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:17.563450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:17.563450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:17.563450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:17.563450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:17.563450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:17.563450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:17.563450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:17.563450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:17.563450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:17.563450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:17.563450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:17.563450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:17.563450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:17.563450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:17.563450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:17.563450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:17.564442 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:17.564442 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:17.564442 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:17.567789 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 436/851] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:17.569586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995b457-2a2c-49c9-b6f0-ff2ca9970c6c None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:17.638220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:17.658018 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3f154-a445-4490-96c8-dffff205c73a None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:17.685553 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995b457-2a2c-49c9-b6f0-ff2ca9970c6c None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:17.693826 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a6ad0-ac06-440f-a04c-1b0d5d084884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:17.694545 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a6ad0-ac06-440f-a04c-1b0d5d084884 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:17.694895 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a6ad0-ac06-440f-a04c-1b0d5d084884 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:17.695238 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a6ad0-ac06-440f-a04c-1b0d5d084884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:17.706747 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a6ad0-ac06-440f-a04c-1b0d5d084884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:17.708317 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a6ad0-ac06-440f-a04c-1b0d5d084884 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:17.712933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:17.722576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-629a6ad0-ac06-440f-a04c-1b0d5d084884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:17.722576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:17.722576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:17.722576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:17.722576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:17.722576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:17.722576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:17.722576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:17.722576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:17.722576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:17.722576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:17.722576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:17.722576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:17.722576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:17.722576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:17.722576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:17.723556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:17.723556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:17.723556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:17.723556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:17.723556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:17.723556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:17.723556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:17.723556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:17.723556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:17.723556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:17.723556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:17.723556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:17.723556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:17.723556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:17.723556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:17.723556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:17.723556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:17.724533 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:17.724533 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:17.724533 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:17.724533 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 416/852] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:17.725243 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193eb28c-abb6-4005-b83e-3265ecd7a5bf None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:17.725342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193eb28c-abb6-4005-b83e-3265ecd7a5bf None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:17.725541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193eb28c-abb6-4005-b83e-3265ecd7a5bf None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:17.726926 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 417/853] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:17.730317 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c41dc7-c22e-40a0-89fb-548cb8701ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:17.730996 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c41dc7-c22e-40a0-89fb-548cb8701ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:17.731342 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c41dc7-c22e-40a0-89fb-548cb8701ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:17.731647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c41dc7-c22e-40a0-89fb-548cb8701ee3 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:17.732132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a8045-6915-4adc-873f-0658af859158 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:17.732322 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c41dc7-c22e-40a0-89fb-548cb8701ee3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:17.732434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a8045-6915-4adc-873f-0658af859158 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:17.732619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c41dc7-c22e-40a0-89fb-548cb8701ee3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:17.732715 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a8045-6915-4adc-873f-0658af859158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:17.733967 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c41dc7-c22e-40a0-89fb-548cb8701ee3 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:17.751828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 437/854] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:17.751828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c41dc7-c22e-40a0-89fb-548cb8701ee3 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:17.757373 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-c74a8045-6915-4adc-873f-0658af859158 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:17.758666 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c74a8045-6915-4adc-873f-0658af859158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:17.827727 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 418/855] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:17.828643 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a8045-6915-4adc-873f-0658af859158 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:17.867894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c41dc7-c22e-40a0-89fb-548cb8701ee3 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:17.901025 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:17.905306 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc6a32-1beb-46a6-9226-c7d7f1f42b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:17.906188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc6a32-1beb-46a6-9226-c7d7f1f42b8b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:17.907626 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc6a32-1beb-46a6-9226-c7d7f1f42b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:17.908113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc6a32-1beb-46a6-9226-c7d7f1f42b8b None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:17.908999 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc6a32-1beb-46a6-9226-c7d7f1f42b8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:17.909487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc6a32-1beb-46a6-9226-c7d7f1f42b8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:17.911168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc6a32-1beb-46a6-9226-c7d7f1f42b8b None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:17.923558 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 438/856] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:17.930790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc6a32-1beb-46a6-9226-c7d7f1f42b8b None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:17.942252 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a8045-6915-4adc-873f-0658af859158 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:17.989155 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf96f08-7112-4864-9c35-5fdbd37a3343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:17.990234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf96f08-7112-4864-9c35-5fdbd37a3343 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:17.990723 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:17.990806 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf96f08-7112-4864-9c35-5fdbd37a3343 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:17.991341 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf96f08-7112-4864-9c35-5fdbd37a3343 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:18.000764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf96f08-7112-4864-9c35-5fdbd37a3343 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:18.002741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf96f08-7112-4864-9c35-5fdbd37a3343 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:18.019464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-5bf96f08-7112-4864-9c35-5fdbd37a3343 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:18.019464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:18.019464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:18.019464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:18.019464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:18.019464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:18.019464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:18.019464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:18.019464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:18.019464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:18.019464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:18.019464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:18.019464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:00:18.019464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:00:18.019464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:00:18.020527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:00:18.020527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:18.020527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:18.020527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:18.020527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:18.020527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:18.020527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:18.020527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:18.020527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:18.020527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:18.020527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:18.020527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:18.020527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:18.020527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:18.020527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:18.020527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:18.020527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:18.021972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:00:18.021972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:00:18.021972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:00:18.021972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:00:18.021972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:00:18.021972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:00:18.021972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:18.021972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:18.021972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 419/857] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:18.028342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf96f08-7112-4864-9c35-5fdbd37a3343 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:18.055878 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc6a32-1beb-46a6-9226-c7d7f1f42b8b None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:18.094839 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd5023-38cf-4310-a976-52f6b6993685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:18.095506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd5023-38cf-4310-a976-52f6b6993685 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:18.095786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd5023-38cf-4310-a976-52f6b6993685 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:18.096056 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd5023-38cf-4310-a976-52f6b6993685 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:18.104921 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd5023-38cf-4310-a976-52f6b6993685 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:18.106650 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd5023-38cf-4310-a976-52f6b6993685 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:18.119078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-c6dd5023-38cf-4310-a976-52f6b6993685 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:00:18.119078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:18.119078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:18.119078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:18.119078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:18.119078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:18.119078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:18.119078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:18.119078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:18.119078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:18.119078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:18.119078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:18.119078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:00:18.119078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:00:18.119078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:00:18.119078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:00:18.120283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:18.120283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:18.120283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:00:18.120283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:18.120283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:00:18.120283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:18.120283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:18.120283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:18.120283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:18.120283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:18.120283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:18.120283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:18.120283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:18.120283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:18.120283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:18.120283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:18.120283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:18.120283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:18.121397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:18.121397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:18.121397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:00:18.121397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:00:18.121397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:00:18.121397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:00:18.121397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:00:18.121397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:00:18.121397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:00:18.121397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:18.124937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 439/858] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 23 07:00:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:18.126416 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd5023-38cf-4310-a976-52f6b6993685 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:18.163402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf96f08-7112-4864-9c35-5fdbd37a3343 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:18.190479 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:18.200122 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf538ad0-3c57-4171-9ab6-65d47fb93b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:18.200738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf538ad0-3c57-4171-9ab6-65d47fb93b64 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:18.201016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf538ad0-3c57-4171-9ab6-65d47fb93b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:18.201432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf538ad0-3c57-4171-9ab6-65d47fb93b64 None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:18.202006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf538ad0-3c57-4171-9ab6-65d47fb93b64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:18.202288 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf538ad0-3c57-4171-9ab6-65d47fb93b64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:18.203761 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf538ad0-3c57-4171-9ab6-65d47fb93b64 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:18.223015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 420/859] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:00:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:18.223895 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf538ad0-3c57-4171-9ab6-65d47fb93b64 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:18.244696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd5023-38cf-4310-a976-52f6b6993685 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:18.248072 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:18.283879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c52179-67e2-445e-83dc-d59273a1a5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:18.284487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c52179-67e2-445e-83dc-d59273a1a5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:18.284832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c52179-67e2-445e-83dc-d59273a1a5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:18.285026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c52179-67e2-445e-83dc-d59273a1a5b8 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:18.285753 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c52179-67e2-445e-83dc-d59273a1a5b8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:18.286066 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c52179-67e2-445e-83dc-d59273a1a5b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:18.288246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c52179-67e2-445e-83dc-d59273a1a5b8 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:18.311627 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 440/860] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Aug 23 07:00:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:18.311627 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c52179-67e2-445e-83dc-d59273a1a5b8 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:18.338467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf538ad0-3c57-4171-9ab6-65d47fb93b64 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:18.376084 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca32f9-5f56-4a84-84eb-b0f99080f035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:18.376721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca32f9-5f56-4a84-84eb-b0f99080f035 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:18.377003 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca32f9-5f56-4a84-84eb-b0f99080f035 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:18.377275 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca32f9-5f56-4a84-84eb-b0f99080f035 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:18.377856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca32f9-5f56-4a84-84eb-b0f99080f035 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:18.379232 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca32f9-5f56-4a84-84eb-b0f99080f035 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:18.389150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-dcca32f9-5f56-4a84-84eb-b0f99080f035 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:00:18.389150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:18.389150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:18.389150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:18.389150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:18.389150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:18.389150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:18.389150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:18.389150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:00:18.389150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:18.389150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:18.389150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:18.389150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:18.389150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:18.389150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:18.389150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:18.390350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:18.390350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:18.390350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:18.390350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:18.390350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:18.390350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:18.390350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:18.390350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:18.390350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:18.390350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:18.390350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:18.390350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:18.390350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:18.390350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:18.390350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:18.390350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:18.390350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:18.391476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:18.391476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:18.391476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:18.391476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:18.391476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:18.391476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:18.391476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:18.391476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:18.391476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:18.391476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:18.391476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:18.391476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:18.391476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:00:18.391476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:18.391476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 421/861] 10.222.0.17 () {56 vars in 1132 bytes} [Tue Aug 23 07:00:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:18.392279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca32f9-5f56-4a84-84eb-b0f99080f035 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:18.421409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c52179-67e2-445e-83dc-d59273a1a5b8 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:18.450208 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:18.458927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257af0a-5b6d-4393-9555-3a8a6d4b3055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:18.459566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257af0a-5b6d-4393-9555-3a8a6d4b3055 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:18.459849 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257af0a-5b6d-4393-9555-3a8a6d4b3055 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:18.460121 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257af0a-5b6d-4393-9555-3a8a6d4b3055 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:18.460851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2257af0a-5b6d-4393-9555-3a8a6d4b3055 None admin] RBAC: Adding query filter params (role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=fd88ec8b477443af95bff9af03471adf) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:18.461159 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2257af0a-5b6d-4393-9555-3a8a6d4b3055 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:18.462585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2257af0a-5b6d-4393-9555-3a8a6d4b3055 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:18.475450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 441/862] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:00:18 2022] GET /identity/v3/role_assignments?user.id=fd88ec8b477443af95bff9af03471adf&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:18.475450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257af0a-5b6d-4393-9555-3a8a6d4b3055 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:18.505931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:18.518628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca32f9-5f56-4a84-84eb-b0f99080f035 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:18.564957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbea69-9bdb-49c3-b761-4afe70bb3915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:18.565612 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbea69-9bdb-49c3-b761-4afe70bb3915 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:18.566576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbea69-9bdb-49c3-b761-4afe70bb3915 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:18.566856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbea69-9bdb-49c3-b761-4afe70bb3915 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:18.594626 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257af0a-5b6d-4393-9555-3a8a6d4b3055 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:18.637791 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a703c-cbd3-4ef8-963e-ee72a439f869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:18.638562 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a703c-cbd3-4ef8-963e-ee72a439f869 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:18.638819 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a703c-cbd3-4ef8-963e-ee72a439f869 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:18.639441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a703c-cbd3-4ef8-963e-ee72a439f869 None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:18.640100 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a703c-cbd3-4ef8-963e-ee72a439f869 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:18.640409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a703c-cbd3-4ef8-963e-ee72a439f869 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:18.642854 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a703c-cbd3-4ef8-963e-ee72a439f869 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:18.649396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 442/863] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 23 07:00:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:18.650764 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e91ff-8882-4071-8768-ca7f81244885 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:18.650875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e91ff-8882-4071-8768-ca7f81244885 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:18.651154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e91ff-8882-4071-8768-ca7f81244885 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:18.652682 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 443/864] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:18.653852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbea69-9bdb-49c3-b761-4afe70bb3915 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:18.655592 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbea69-9bdb-49c3-b761-4afe70bb3915 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:18.657435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e91ff-8882-4071-8768-ca7f81244885 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:18.720828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:18.761048 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:18.777304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 422/865] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:00:18 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:00:18.777522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e91ff-8882-4071-8768-ca7f81244885 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:18.777610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff2fb7-8537-4fcd-8a83-3012d9ca155a None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:18.777610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff2fb7-8537-4fcd-8a83-3012d9ca155a None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:18.777610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff2fb7-8537-4fcd-8a83-3012d9ca155a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:18.792821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-deff2fb7-8537-4fcd-8a83-3012d9ca155a None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:18.794021 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-deff2fb7-8537-4fcd-8a83-3012d9ca155a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:18.813236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9767c-0454-4208-b3a4-2c69b6d5c9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:18.813960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9767c-0454-4208-b3a4-2c69b6d5c9a5 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:18.814333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9767c-0454-4208-b3a4-2c69b6d5c9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:18.814607 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9767c-0454-4208-b3a4-2c69b6d5c9a5 None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:18.815232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9767c-0454-4208-b3a4-2c69b6d5c9a5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:18.815562 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9767c-0454-4208-b3a4-2c69b6d5c9a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:18.817113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9767c-0454-4208-b3a4-2c69b6d5c9a5 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:18.831451 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 444/866] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 23 07:00:18 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:18.831451 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9767c-0454-4208-b3a4-2c69b6d5c9a5 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:18.880383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 423/867] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:18.950750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9767c-0454-4208-b3a4-2c69b6d5c9a5 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:18.988948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9682f9-4a32-4e59-bdcb-8c016659fad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:18.989514 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9682f9-4a32-4e59-bdcb-8c016659fad3 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:18.989969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9682f9-4a32-4e59-bdcb-8c016659fad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:18.990360 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9682f9-4a32-4e59-bdcb-8c016659fad3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:19.003067 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9682f9-4a32-4e59-bdcb-8c016659fad3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:19.004585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9682f9-4a32-4e59-bdcb-8c016659fad3 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:19.020484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-ae9682f9-4a32-4e59-bdcb-8c016659fad3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:19.020484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:19.020484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:19.020484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:19.020484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:19.020484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:19.020484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:19.020484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:19.020484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:19.020484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:19.020484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:19.020484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:19.020484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:19.020484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:19.020484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:19.020484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:19.021586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:19.021586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:19.021586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:19.021586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:19.021586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:19.021586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:19.021586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:19.021586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:19.021586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:19.021586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:19.021586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:19.021586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:19.021586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:19.021586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:19.021586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:19.021586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:19.021586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:19.022482 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:19.022482 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:19.022482 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:19.022482 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 445/868] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:19.029371 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff2fb7-8537-4fcd-8a83-3012d9ca155a None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:19.119777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:19.138767 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff2fb7-8537-4fcd-8a83-3012d9ca155a None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:19.149763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9682f9-4a32-4e59-bdcb-8c016659fad3 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:19.159932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:19.175493 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa09343-4024-48b6-b12c-c9f80d9942f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:19.176112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa09343-4024-48b6-b12c-c9f80d9942f7 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:19.176684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa09343-4024-48b6-b12c-c9f80d9942f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:19.176684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa09343-4024-48b6-b12c-c9f80d9942f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:19.177336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa09343-4024-48b6-b12c-c9f80d9942f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:19.177732 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa09343-4024-48b6-b12c-c9f80d9942f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:19.179028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa09343-4024-48b6-b12c-c9f80d9942f7 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:19.191377 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 424/869] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:19.197915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa09343-4024-48b6-b12c-c9f80d9942f7 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:19.275328 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9682f9-4a32-4e59-bdcb-8c016659fad3 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:19.309316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa09343-4024-48b6-b12c-c9f80d9942f7 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:19.312653 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec51c5-bf29-46c3-8518-1f0cebe0c67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:19.313433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec51c5-bf29-46c3-8518-1f0cebe0c67d None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:19.313857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec51c5-bf29-46c3-8518-1f0cebe0c67d None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:19.314297 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec51c5-bf29-46c3-8518-1f0cebe0c67d None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:19.315101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec51c5-bf29-46c3-8518-1f0cebe0c67d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:19.316660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec51c5-bf29-46c3-8518-1f0cebe0c67d None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:19.336665 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 446/870] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 23 07:00:19 2022] POST /identity/v3/services => generated 251 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:00:19.361934 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1de69-a8e5-4148-a527-28b923bd80de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:19.363346 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1de69-a8e5-4148-a527-28b923bd80de None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:19.363816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1de69-a8e5-4148-a527-28b923bd80de None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:19.364231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1de69-a8e5-4148-a527-28b923bd80de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:19.372906 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1de69-a8e5-4148-a527-28b923bd80de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:19.374856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1de69-a8e5-4148-a527-28b923bd80de None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:19.377356 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:19.389463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-47f1de69-a8e5-4148-a527-28b923bd80de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:19.389463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:19.389463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:19.389463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:19.389463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:19.389463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:19.389463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:19.389463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:19.389463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:19.389463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:19.389463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:19.389463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:19.389463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:00:19.389463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:00:19.389463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:00:19.390630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:00:19.390630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:19.390630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:19.390630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:19.390630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:19.390630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:19.390630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:19.390630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:19.390630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:19.390630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:19.390630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:19.390630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:19.390630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:19.390630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:19.390630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:19.390630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:19.390630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:19.391704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:00:19.391704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:00:19.391704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:00:19.391704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:00:19.391704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:00:19.391704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:00:19.391704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:19.391704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:19.399638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec51c5-bf29-46c3-8518-1f0cebe0c67d None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:19.410067 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 425/871] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:19.477606 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:19.515285 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec51c5-bf29-46c3-8518-1f0cebe0c67d None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:19.550906 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb32ba7-09b4-4b26-914b-4c482568d958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:19.551830 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb32ba7-09b4-4b26-914b-4c482568d958 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:19.552256 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb32ba7-09b4-4b26-914b-4c482568d958 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:19.552769 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb32ba7-09b4-4b26-914b-4c482568d958 None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:19.553605 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb32ba7-09b4-4b26-914b-4c482568d958 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:19.554071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb32ba7-09b4-4b26-914b-4c482568d958 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:19.555667 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb32ba7-09b4-4b26-914b-4c482568d958 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:19.569353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 447/872] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:00:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:19.575761 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1de69-a8e5-4148-a527-28b923bd80de None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:19.680133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1de69-a8e5-4148-a527-28b923bd80de None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:19.717046 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639e570-2a50-4b43-8092-125f2ae14263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:19.717663 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639e570-2a50-4b43-8092-125f2ae14263 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:19.717946 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639e570-2a50-4b43-8092-125f2ae14263 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:19.718219 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639e570-2a50-4b43-8092-125f2ae14263 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:19.718958 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639e570-2a50-4b43-8092-125f2ae14263 None admin] RBAC: Adding query filter params (role.id=2d62b8b8395745c7983c6c8d100d985b, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=b3f1ae88e29f4172a7db7393d517a893) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:19.719267 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639e570-2a50-4b43-8092-125f2ae14263 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:19.720739 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639e570-2a50-4b43-8092-125f2ae14263 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:19.727123 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 426/873] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:00:19 2022] GET /identity/v3/role_assignments?user.id=b3f1ae88e29f4172a7db7393d517a893&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2d62b8b8395745c7983c6c8d100d985b => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:20.523792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad7502-3591-47a1-8ae7-eab14d17d886 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:20.524404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad7502-3591-47a1-8ae7-eab14d17d886 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:20.524404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad7502-3591-47a1-8ae7-eab14d17d886 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:20.525872 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 448/874] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:20.531383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a623c-22cf-4063-a7f9-69743cd0a075 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:20.531790 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a623c-22cf-4063-a7f9-69743cd0a075 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:20.531957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a623c-22cf-4063-a7f9-69743cd0a075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:20.550072 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-7a1a623c-22cf-4063-a7f9-69743cd0a075 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:20.551251 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-7a1a623c-22cf-4063-a7f9-69743cd0a075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:20.612263 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:20.623485 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 427/875] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:20.769666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7d77a-b43b-4d91-b90d-fb98b87d80a0 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:20.770323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7d77a-b43b-4d91-b90d-fb98b87d80a0 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:20.770802 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7d77a-b43b-4d91-b90d-fb98b87d80a0 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:20.772724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 449/876] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:20.779953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e5b9a-9073-4694-865b-2f2acf37cbc1 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:20.780171 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e5b9a-9073-4694-865b-2f2acf37cbc1 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:20.780452 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e5b9a-9073-4694-865b-2f2acf37cbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:20.805954 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-671e5b9a-9073-4694-865b-2f2acf37cbc1 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:20.807225 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-671e5b9a-9073-4694-865b-2f2acf37cbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:20.879830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 428/877] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:20.913577 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7d77a-b43b-4d91-b90d-fb98b87d80a0 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:21.021356 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7d77a-b43b-4d91-b90d-fb98b87d80a0 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:21.048158 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:21.057145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20745463-92c6-4c2a-bc98-94ef6abfd932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:21.057773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20745463-92c6-4c2a-bc98-94ef6abfd932 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:21.058052 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20745463-92c6-4c2a-bc98-94ef6abfd932 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:21.058321 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20745463-92c6-4c2a-bc98-94ef6abfd932 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:21.058913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20745463-92c6-4c2a-bc98-94ef6abfd932 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:21.060366 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20745463-92c6-4c2a-bc98-94ef6abfd932 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:21.069792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-20745463-92c6-4c2a-bc98-94ef6abfd932 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:00:21.069792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:21.069792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:21.069792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:21.069792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:21.069792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:21.069792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:21.069792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:21.069792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:00:21.069792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:21.069792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:21.069792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:21.069792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:21.069792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:21.069792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:21.069792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:21.069792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:21.070771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:21.070771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:21.070771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:21.070771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:21.070771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:21.070771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:21.070771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:21.070771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:21.070771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:21.070771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:21.070771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:21.070771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:21.070771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:21.070771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:21.070771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:21.070771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:21.071642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:21.071642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:21.071642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:21.071642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:21.071642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:21.071642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:21.071642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:21.071642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:21.071642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:21.071642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:21.071642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:21.071642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:21.071642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:00:21.071642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:21.071642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 450/878] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 23 07:00:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:21.077154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e5b9a-9073-4694-865b-2f2acf37cbc1 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:21.157973 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20745463-92c6-4c2a-bc98-94ef6abfd932 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:21.186030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e5b9a-9073-4694-865b-2f2acf37cbc1 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:21.214368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:21.227909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d1554-8890-40ad-afef-21410d42688b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:21.228848 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d1554-8890-40ad-afef-21410d42688b None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:21.229343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d1554-8890-40ad-afef-21410d42688b None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:21.229827 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d1554-8890-40ad-afef-21410d42688b None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:21.230671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d1554-8890-40ad-afef-21410d42688b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:21.231181 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d1554-8890-40ad-afef-21410d42688b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:21.232875 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d1554-8890-40ad-afef-21410d42688b None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:21.238188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 429/879] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 23 07:00:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:21.244141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d1554-8890-40ad-afef-21410d42688b None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:21.287068 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20745463-92c6-4c2a-bc98-94ef6abfd932 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:21.321391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181cbaf-964e-439b-bdbd-03d650a788e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:21.321927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181cbaf-964e-439b-bdbd-03d650a788e6 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:21.322243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181cbaf-964e-439b-bdbd-03d650a788e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:21.322546 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181cbaf-964e-439b-bdbd-03d650a788e6 None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:21.323160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181cbaf-964e-439b-bdbd-03d650a788e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:21.324585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181cbaf-964e-439b-bdbd-03d650a788e6 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:21.342597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 451/880] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 23 07:00:21 2022] POST /identity/v3/services => generated 269 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:00:21.344101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501967f0-38ea-4955-8ea2-9c41a9cf950c None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:21.344388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501967f0-38ea-4955-8ea2-9c41a9cf950c None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:21.344714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501967f0-38ea-4955-8ea2-9c41a9cf950c None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:21.346397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 452/881] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:21.356933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d171e-fdb0-4326-bbbe-c6caead59a51 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:21.357210 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d171e-fdb0-4326-bbbe-c6caead59a51 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:21.357487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d171e-fdb0-4326-bbbe-c6caead59a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:21.372310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d1554-8890-40ad-afef-21410d42688b None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:21.377916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-264d171e-fdb0-4326-bbbe-c6caead59a51 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:21.379153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-264d171e-fdb0-4326-bbbe-c6caead59a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:21.424201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521735e9-e3ca-45cd-ac7a-2c731913c36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:21.424740 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521735e9-e3ca-45cd-ac7a-2c731913c36a None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:21.425105 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521735e9-e3ca-45cd-ac7a-2c731913c36a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:21.425388 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521735e9-e3ca-45cd-ac7a-2c731913c36a None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:21.426023 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521735e9-e3ca-45cd-ac7a-2c731913c36a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:21.426327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521735e9-e3ca-45cd-ac7a-2c731913c36a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:21.428370 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521735e9-e3ca-45cd-ac7a-2c731913c36a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:21.434254 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 430/882] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 23 07:00:21 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:21.448438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:21.475387 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 453/883] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:21.754666 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521735e9-e3ca-45cd-ac7a-2c731913c36a None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:21.770773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22867a0-e720-44d8-9346-8c3b7be50a43 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:21.771377 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22867a0-e720-44d8-9346-8c3b7be50a43 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:21.771491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22867a0-e720-44d8-9346-8c3b7be50a43 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:21.773928 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 454/884] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:21.778740 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cc91f-01bf-4236-8a0e-db3b3248af82 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:21.779023 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cc91f-01bf-4236-8a0e-db3b3248af82 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:21.779299 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cc91f-01bf-4236-8a0e-db3b3248af82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:21.797827 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-ae1cc91f-01bf-4236-8a0e-db3b3248af82 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:21.798961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ae1cc91f-01bf-4236-8a0e-db3b3248af82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:21.812956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:21.882225 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521735e9-e3ca-45cd-ac7a-2c731913c36a None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:21.883552 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 455/885] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:21.919584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e59da-c545-4ff8-8195-5387f0ef17c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:21.920555 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e59da-c545-4ff8-8195-5387f0ef17c9 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:21.921036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e59da-c545-4ff8-8195-5387f0ef17c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:21.921510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e59da-c545-4ff8-8195-5387f0ef17c9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:21.922331 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e59da-c545-4ff8-8195-5387f0ef17c9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:21.924009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e59da-c545-4ff8-8195-5387f0ef17c9 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:21.933190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-ad4e59da-c545-4ff8-8195-5387f0ef17c9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:00:21.933190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:21.933190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:21.933190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:21.933190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:21.933190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:21.933190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:21.933190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:21.933190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:00:21.933190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:21.933190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:21.933190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:21.933190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:21.933190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:21.933190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:21.933190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:21.933190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:21.934189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:21.934189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:21.934189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:21.934189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:21.934189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:21.934189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:21.934189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:21.934189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:21.934189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:21.934189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:21.934189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:21.934189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:21.934189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:21.934189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:21.934189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:21.934189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:21.935030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:21.935030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:21.935030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:21.935030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:21.935030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:21.935030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:21.935030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:21.935030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:21.935030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:21.935030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:21.935030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:21.935030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:21.935030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:00:21.935030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:21.937739 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 431/886] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 07:00:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:21.943790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cc91f-01bf-4236-8a0e-db3b3248af82 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:22.046410 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cc91f-01bf-4236-8a0e-db3b3248af82 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:22.058305 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:22.083865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1718df-a35d-4eb9-a367-27cdb32cde48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:22.084960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1718df-a35d-4eb9-a367-27cdb32cde48 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:22.085453 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1718df-a35d-4eb9-a367-27cdb32cde48 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:22.085937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1718df-a35d-4eb9-a367-27cdb32cde48 None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:22.086768 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1718df-a35d-4eb9-a367-27cdb32cde48 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:22.087279 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1718df-a35d-4eb9-a367-27cdb32cde48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:22.088879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1718df-a35d-4eb9-a367-27cdb32cde48 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:22.094190 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 456/887] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 07:00:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:22.100109 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e59da-c545-4ff8-8195-5387f0ef17c9 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:22.157134 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1718df-a35d-4eb9-a367-27cdb32cde48 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:22.225606 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e59da-c545-4ff8-8195-5387f0ef17c9 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:22.238620 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:22.272915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec714304-aba5-4693-b505-bf449b09869d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:22.273785 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec714304-aba5-4693-b505-bf449b09869d None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:22.274070 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec714304-aba5-4693-b505-bf449b09869d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:22.274349 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec714304-aba5-4693-b505-bf449b09869d None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:22.275063 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec714304-aba5-4693-b505-bf449b09869d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:22.275650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec714304-aba5-4693-b505-bf449b09869d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:22.277730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec714304-aba5-4693-b505-bf449b09869d None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:22.279540 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1718df-a35d-4eb9-a367-27cdb32cde48 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:22.289490 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 432/888] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 07:00:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:22.290450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec714304-aba5-4693-b505-bf449b09869d None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:22.319469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf1c7d-8f01-446a-a166-751c6fd544bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:22.320479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf1c7d-8f01-446a-a166-751c6fd544bd None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:22.320971 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf1c7d-8f01-446a-a166-751c6fd544bd None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:22.321438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf1c7d-8f01-446a-a166-751c6fd544bd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:22.333104 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf1c7d-8f01-446a-a166-751c6fd544bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:00:22.338931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-60bf1c7d-8f01-446a-a166-751c6fd544bd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:00:22.338931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:22.338931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:22.338931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:22.338931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:22.338931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:22.338931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:22.338931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:22.338931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:22.338931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:22.338931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:22.338931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:22.338931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:00:22.338931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:00:22.338931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:00:22.338931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:00:22.339870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:00:22.339870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:00:22.339870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:00:22.339870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:00:22.339870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:22.339870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:22.339870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:22.339870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:22.339870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:22.339870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:22.339870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:22.339870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:22.339870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:22.339870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:22.339870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:22.339870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:22.340710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:22.340710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:22.340710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:22.340710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:22.340710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:00:22.340710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:00:22.340710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:00:22.340710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:00:22.340710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:00:22.340710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:00:22.340710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:00:22.340710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:22.348053 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 457/889] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Aug 23 07:00:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:22.350989 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf1c7d-8f01-446a-a166-751c6fd544bd None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:22.399891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:22.415436 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec714304-aba5-4693-b505-bf449b09869d None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:22.450814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ac739-d375-448a-9dce-5dc5228cc238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:22.451444 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ac739-d375-448a-9dce-5dc5228cc238 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:22.451726 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ac739-d375-448a-9dce-5dc5228cc238 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:22.451997 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ac739-d375-448a-9dce-5dc5228cc238 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:22.452700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ac739-d375-448a-9dce-5dc5228cc238 None admin] RBAC: Adding query filter params (interface=public, service_id=fa7f50370e2741669210c1aa28258291, region_id=RegionOne) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:22.453005 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ac739-d375-448a-9dce-5dc5228cc238 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:22.454375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ac739-d375-448a-9dce-5dc5228cc238 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:22.460380 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 433/890] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Aug 23 07:00:22 2022] GET /identity/v3/endpoints?service_id=fa7f50370e2741669210c1aa28258291&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:22.472464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ac739-d375-448a-9dce-5dc5228cc238 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:22.473102 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf1c7d-8f01-446a-a166-751c6fd544bd None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:22.515089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80925efd-4108-47ba-8075-1547d4ba2a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:22.516077 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80925efd-4108-47ba-8075-1547d4ba2a44 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:22.516497 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80925efd-4108-47ba-8075-1547d4ba2a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:22.516870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80925efd-4108-47ba-8075-1547d4ba2a44 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:22.517626 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80925efd-4108-47ba-8075-1547d4ba2a44 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:22.518022 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80925efd-4108-47ba-8075-1547d4ba2a44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:22.519658 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80925efd-4108-47ba-8075-1547d4ba2a44 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:22.526403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:22.534685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 458/891] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Aug 23 07:00:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:22.536924 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80925efd-4108-47ba-8075-1547d4ba2a44 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:22.599192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ac739-d375-448a-9dce-5dc5228cc238 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:22.639770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443ddd0-a7ec-4570-a993-b1bec7d1f630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:22.640515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443ddd0-a7ec-4570-a993-b1bec7d1f630 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:22.640891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443ddd0-a7ec-4570-a993-b1bec7d1f630 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:22.641259 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443ddd0-a7ec-4570-a993-b1bec7d1f630 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:22.641982 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443ddd0-a7ec-4570-a993-b1bec7d1f630 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:22.643455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443ddd0-a7ec-4570-a993-b1bec7d1f630 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:22.649726 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:22.659669 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 434/892] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Aug 23 07:00:22 2022] GET /identity/v3/services => generated 1268 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:00:22.664035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80925efd-4108-47ba-8075-1547d4ba2a44 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:22.711901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b90e58-8a23-4f91-a26b-3bb366050a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:22.712551 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b90e58-8a23-4f91-a26b-3bb366050a94 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:22.712873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b90e58-8a23-4f91-a26b-3bb366050a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:22.713142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b90e58-8a23-4f91-a26b-3bb366050a94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:22.725802 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b90e58-8a23-4f91-a26b-3bb366050a94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:22.727281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b90e58-8a23-4f91-a26b-3bb366050a94 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:22.743176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-76b90e58-8a23-4f91-a26b-3bb366050a94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:22.743176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:22.743176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:22.743176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:22.743176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:22.743176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:22.743176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:22.743176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:22.743176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:22.743176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:22.743176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:22.743176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:22.743176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:22.743176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:22.743176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:22.743176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:22.744247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:22.744247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:22.744247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:22.744247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:22.744247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:22.744247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:22.744247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:22.744247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:22.744247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:22.744247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:22.744247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:22.744247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:22.744247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:22.744247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:22.744247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:22.744247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:22.744247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:22.745128 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:22.745128 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:22.745128 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:22.747503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 459/893] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:22.750844 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443ddd0-a7ec-4570-a993-b1bec7d1f630 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:22.865161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443ddd0-a7ec-4570-a993-b1bec7d1f630 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:22.867830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:22.903845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab9ffe-933d-42d9-9656-173c0563a1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:22.904898 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab9ffe-933d-42d9-9656-173c0563a1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:22.905416 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab9ffe-933d-42d9-9656-173c0563a1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:22.905888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab9ffe-933d-42d9-9656-173c0563a1b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:22.906736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab9ffe-933d-42d9-9656-173c0563a1b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:22.907236 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab9ffe-933d-42d9-9656-173c0563a1b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:22.908958 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab9ffe-933d-42d9-9656-173c0563a1b0 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:22.921843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 435/894] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:22.927264 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b90e58-8a23-4f91-a26b-3bb366050a94 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:23.031902 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b90e58-8a23-4f91-a26b-3bb366050a94 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:23.069869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d3f6e-f7c8-4041-ab4c-403a1628c7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:23.070952 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d3f6e-f7c8-4041-ab4c-403a1628c7a1 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:23.071457 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d3f6e-f7c8-4041-ab4c-403a1628c7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:23.071932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d3f6e-f7c8-4041-ab4c-403a1628c7a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:23.080843 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d3f6e-f7c8-4041-ab4c-403a1628c7a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:23.082627 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d3f6e-f7c8-4041-ab4c-403a1628c7a1 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:23.095027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-0a9d3f6e-f7c8-4041-ab4c-403a1628c7a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:00:23.095027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:23.095027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:23.095027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:23.095027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:23.095027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:23.095027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:23.095027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:23.095027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:23.095027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:23.095027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:23.095027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:23.095027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:00:23.095027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:00:23.095027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:00:23.095027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:00:23.096019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:23.096019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:23.096019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:00:23.096019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:23.096019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:00:23.096019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:23.096019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:23.096019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:23.096019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:23.096019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:23.096019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:23.096019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:23.096019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:23.096019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:23.096019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:23.096019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:23.096019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:23.096019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:23.097003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:23.097003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:23.097003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:00:23.097003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:00:23.097003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:00:23.097003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:00:23.097003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:00:23.097003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:00:23.097003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:00:23.097003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:23.099573 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 460/895] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 23 07:00:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:23.108522 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab9ffe-933d-42d9-9656-173c0563a1b0 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:23.215811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab9ffe-933d-42d9-9656-173c0563a1b0 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:23.251144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58118cbf-ca83-4654-89db-233d76b70e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:23.251870 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58118cbf-ca83-4654-89db-233d76b70e5e None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:23.252145 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58118cbf-ca83-4654-89db-233d76b70e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:23.252413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58118cbf-ca83-4654-89db-233d76b70e5e None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:23.253206 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58118cbf-ca83-4654-89db-233d76b70e5e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:23.253512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58118cbf-ca83-4654-89db-233d76b70e5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:23.254970 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58118cbf-ca83-4654-89db-233d76b70e5e None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:23.277325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 436/896] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Aug 23 07:00:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:23.279788 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d3f6e-f7c8-4041-ab4c-403a1628c7a1 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:23.292202 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:23.324380 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f2159-95fd-461c-9878-f1f8c625c54f None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:23.324727 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f2159-95fd-461c-9878-f1f8c625c54f None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:23.324929 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f2159-95fd-461c-9878-f1f8c625c54f None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:23.326527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 437/897] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:00:23.332172 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bb048-5414-44e3-9c0c-5c37d039e594 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:23.332455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bb048-5414-44e3-9c0c-5c37d039e594 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:23.332732 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bb048-5414-44e3-9c0c-5c37d039e594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:23.352791 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-be1bb048-5414-44e3-9c0c-5c37d039e594 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:23.354112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-be1bb048-5414-44e3-9c0c-5c37d039e594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:23.373883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:23.407788 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d3f6e-f7c8-4041-ab4c-403a1628c7a1 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:23.439202 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 438/898] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:23.440720 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb7c6a-57ff-4183-a6f4-4fa371b091d7 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:23.441068 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb7c6a-57ff-4183-a6f4-4fa371b091d7 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:23.441410 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb7c6a-57ff-4183-a6f4-4fa371b091d7 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:23.442831 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 439/899] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:23.448474 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28c2a1-6bfc-4d5a-9844-a82f9bf7f5ca None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:23.448762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28c2a1-6bfc-4d5a-9844-a82f9bf7f5ca None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:23.449038 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28c2a1-6bfc-4d5a-9844-a82f9bf7f5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:23.450021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb57d6-75cc-472e-bdb8-4120ec6dd750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:23.450860 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb57d6-75cc-472e-bdb8-4120ec6dd750 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:23.451330 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb57d6-75cc-472e-bdb8-4120ec6dd750 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:23.451895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb57d6-75cc-472e-bdb8-4120ec6dd750 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:23.468049 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-2d28c2a1-6bfc-4d5a-9844-a82f9bf7f5ca None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:23.469510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2d28c2a1-6bfc-4d5a-9844-a82f9bf7f5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:23.532748 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb57d6-75cc-472e-bdb8-4120ec6dd750 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:23.535301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb57d6-75cc-472e-bdb8-4120ec6dd750 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:23.551561 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 440/900] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:23.571602 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:23.651516 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 461/901] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:00:23 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:00:23.655354 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28c2a1-6bfc-4d5a-9844-a82f9bf7f5ca None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:23.727250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb57d6-75cc-472e-bdb8-4120ec6dd750 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:23.745593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:23.763549 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28c2a1-6bfc-4d5a-9844-a82f9bf7f5ca None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:23.798947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251d323-b905-4eb0-9dab-491bc877acb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:23.799716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251d323-b905-4eb0-9dab-491bc877acb7 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:23.800100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251d323-b905-4eb0-9dab-491bc877acb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:23.800372 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251d323-b905-4eb0-9dab-491bc877acb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:23.811441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0251d323-b905-4eb0-9dab-491bc877acb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:23.813136 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0251d323-b905-4eb0-9dab-491bc877acb7 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:23.829304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-0251d323-b905-4eb0-9dab-491bc877acb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:23.829304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:23.829304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:23.829304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:23.829304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:23.829304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:23.829304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:23.829304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:23.829304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:23.829304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:23.829304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:23.829304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:23.829304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:23.829304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:23.829304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:23.829304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:23.830593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:23.830593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:23.830593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:23.830593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:23.830593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:23.830593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:23.830593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:23.830593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:23.830593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:23.830593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:23.830593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:23.830593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:23.830593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:23.830593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:23.830593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:23.830593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:23.830593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:23.831486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:23.831486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:23.831486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:23.831486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 441/902] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:23.836921 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251d323-b905-4eb0-9dab-491bc877acb7 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:23.842749 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb57d6-75cc-472e-bdb8-4120ec6dd750 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:23.858686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:23.881263 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1f832-4201-41a3-a223-74cdc0b9bffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:23.881903 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1f832-4201-41a3-a223-74cdc0b9bffa None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:23.882179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1f832-4201-41a3-a223-74cdc0b9bffa None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:23.882448 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1f832-4201-41a3-a223-74cdc0b9bffa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:23.883030 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1f832-4201-41a3-a223-74cdc0b9bffa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:23.884643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1f832-4201-41a3-a223-74cdc0b9bffa None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:23.895329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-ebc1f832-4201-41a3-a223-74cdc0b9bffa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:00:23.895329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:23.895329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:23.895329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:23.895329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:23.895329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:23.895329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:23.895329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:23.895329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:00:23.895329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:23.895329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:23.895329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:23.895329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:23.895329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:23.895329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:23.895329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:23.896490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:23.896490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:23.896490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:23.896490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:23.896490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:23.896490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:23.896490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:23.896490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:23.896490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:23.896490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:23.896490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:23.896490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:23.896490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:23.896490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:23.896490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:23.896490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:23.896490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:23.897597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:23.897597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:23.897597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:23.897597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:23.897597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:23.897597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:23.897597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:23.897597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:23.897597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:23.897597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:23.897597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:23.897597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:23.897597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:00:23.897597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:23.900690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 462/903] 10.222.0.17 () {56 vars in 1132 bytes} [Tue Aug 23 07:00:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:23.901877 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1f832-4201-41a3-a223-74cdc0b9bffa None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:23.949389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251d323-b905-4eb0-9dab-491bc877acb7 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:23.980977 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:23.987531 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ea6d0-4621-46f5-92c9-4103ae6a55f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:23.988370 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ea6d0-4621-46f5-92c9-4103ae6a55f0 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:23.988813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ea6d0-4621-46f5-92c9-4103ae6a55f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:23.989223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ea6d0-4621-46f5-92c9-4103ae6a55f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:23.990032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ea6d0-4621-46f5-92c9-4103ae6a55f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:23.990621 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ea6d0-4621-46f5-92c9-4103ae6a55f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:23.992336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ea6d0-4621-46f5-92c9-4103ae6a55f0 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:24.004796 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 442/904] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:24.005786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1f832-4201-41a3-a223-74cdc0b9bffa None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:24.006373 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ea6d0-4621-46f5-92c9-4103ae6a55f0 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:24.045123 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b3394-e9a6-4a97-9afe-b6343498ce8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:24.045729 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b3394-e9a6-4a97-9afe-b6343498ce8f None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:24.046003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b3394-e9a6-4a97-9afe-b6343498ce8f None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:24.046271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b3394-e9a6-4a97-9afe-b6343498ce8f None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:24.046913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b3394-e9a6-4a97-9afe-b6343498ce8f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:24.048418 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b3394-e9a6-4a97-9afe-b6343498ce8f None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:24.071684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 463/905] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 23 07:00:23 2022] POST /identity/v3/services => generated 241 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:00:24.071684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b3394-e9a6-4a97-9afe-b6343498ce8f None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:24.115881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:24.129502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ea6d0-4621-46f5-92c9-4103ae6a55f0 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:24.182638 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad490d-ddd8-49c3-8e0e-00dc14a46afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:24.183747 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad490d-ddd8-49c3-8e0e-00dc14a46afc None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:24.184199 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad490d-ddd8-49c3-8e0e-00dc14a46afc None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:24.184750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad490d-ddd8-49c3-8e0e-00dc14a46afc None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:24.185616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad490d-ddd8-49c3-8e0e-00dc14a46afc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:24.186092 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad490d-ddd8-49c3-8e0e-00dc14a46afc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:24.187833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad490d-ddd8-49c3-8e0e-00dc14a46afc None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:24.198897 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 443/906] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 23 07:00:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:24.200650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad490d-ddd8-49c3-8e0e-00dc14a46afc None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:24.212449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b3394-e9a6-4a97-9afe-b6343498ce8f None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:24.260354 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c54d15-8ae5-4997-9e8d-4b1269ae16ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:24.260909 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c54d15-8ae5-4997-9e8d-4b1269ae16ce None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:24.261189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c54d15-8ae5-4997-9e8d-4b1269ae16ce None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:24.261461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c54d15-8ae5-4997-9e8d-4b1269ae16ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:24.262481 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:24.270475 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c54d15-8ae5-4997-9e8d-4b1269ae16ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:24.272627 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c54d15-8ae5-4997-9e8d-4b1269ae16ce None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:24.287744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-b6c54d15-8ae5-4997-9e8d-4b1269ae16ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:24.287744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:24.287744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:24.287744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:24.287744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:24.287744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:24.287744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:24.287744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:24.287744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:24.287744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:24.287744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:24.287744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:24.287744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:00:24.287744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:00:24.287744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:00:24.288700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:00:24.288700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:24.288700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:24.288700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:24.288700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:24.288700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:24.288700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:24.288700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:24.288700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:24.288700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:24.288700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:24.288700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:24.288700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:24.288700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:24.288700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:24.288700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:24.288700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:24.289586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:00:24.289586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:00:24.289586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:00:24.289586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:00:24.289586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:00:24.289586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:00:24.289586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:24.289586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:24.289586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 464/907] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:24.294505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c54d15-8ae5-4997-9e8d-4b1269ae16ce None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:24.323596 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad490d-ddd8-49c3-8e0e-00dc14a46afc None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:24.358841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8f95d-dad2-4a47-9470-11a423d5dca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:24.359536 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8f95d-dad2-4a47-9470-11a423d5dca8 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:24.359830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8f95d-dad2-4a47-9470-11a423d5dca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:24.360159 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8f95d-dad2-4a47-9470-11a423d5dca8 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:24.360847 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8f95d-dad2-4a47-9470-11a423d5dca8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:24.361157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8f95d-dad2-4a47-9470-11a423d5dca8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:24.362557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8f95d-dad2-4a47-9470-11a423d5dca8 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:24.367853 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 444/908] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 23 07:00:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:24.374048 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8f95d-dad2-4a47-9470-11a423d5dca8 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:24.420402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:24.433203 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c54d15-8ae5-4997-9e8d-4b1269ae16ce None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:24.480065 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce334ce-97c3-4e99-b80b-a4016fc2848a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:24.480680 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce334ce-97c3-4e99-b80b-a4016fc2848a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:24.480954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce334ce-97c3-4e99-b80b-a4016fc2848a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:24.481767 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce334ce-97c3-4e99-b80b-a4016fc2848a None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:24.482469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce334ce-97c3-4e99-b80b-a4016fc2848a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:24.482776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce334ce-97c3-4e99-b80b-a4016fc2848a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:24.484764 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce334ce-97c3-4e99-b80b-a4016fc2848a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:24.495843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8f95d-dad2-4a47-9470-11a423d5dca8 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:24.501290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 465/909] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:00:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:24.507121 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce334ce-97c3-4e99-b80b-a4016fc2848a None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:24.518226 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:24.537983 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ae9c0-788a-4362-8c64-6b5f0f10f445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:24.538840 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ae9c0-788a-4362-8c64-6b5f0f10f445 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:24.539333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ae9c0-788a-4362-8c64-6b5f0f10f445 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:24.539823 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ae9c0-788a-4362-8c64-6b5f0f10f445 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:24.540706 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ae9c0-788a-4362-8c64-6b5f0f10f445 None admin] RBAC: Adding query filter params (interface=public, service_id=1eeec9ea49e943dcaf90e048baec03b7, region_id=RegionOne) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:24.541192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ae9c0-788a-4362-8c64-6b5f0f10f445 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:24.542807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ae9c0-788a-4362-8c64-6b5f0f10f445 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:24.549070 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 445/910] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Aug 23 07:00:24 2022] GET /identity/v3/endpoints?service_id=1eeec9ea49e943dcaf90e048baec03b7&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:24.562554 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ae9c0-788a-4362-8c64-6b5f0f10f445 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:24.625607 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce334ce-97c3-4e99-b80b-a4016fc2848a None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:24.664003 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ae9c0-788a-4362-8c64-6b5f0f10f445 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:24.664199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2171ec-7ec3-4865-be13-be8275015717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:24.664616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2171ec-7ec3-4865-be13-be8275015717 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:24.664879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2171ec-7ec3-4865-be13-be8275015717 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:24.665145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2171ec-7ec3-4865-be13-be8275015717 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:24.665868 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2171ec-7ec3-4865-be13-be8275015717 None admin] RBAC: Adding query filter params (role.id=6d1af19135484d85af7accf2fdeb3cab, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=b3f1ae88e29f4172a7db7393d517a893) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:24.666171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2171ec-7ec3-4865-be13-be8275015717 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:24.667633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2171ec-7ec3-4865-be13-be8275015717 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:24.673698 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 466/911] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:00:24 2022] GET /identity/v3/role_assignments?user.id=b3f1ae88e29f4172a7db7393d517a893&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=6d1af19135484d85af7accf2fdeb3cab => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:24.699905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02126e-2108-4888-8908-14cfe09f6f7a None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:24.700235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02126e-2108-4888-8908-14cfe09f6f7a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:24.700520 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02126e-2108-4888-8908-14cfe09f6f7a None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:24.702103 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 467/912] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:00:24.703610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e86ec-3cdb-4881-a5c5-572373138c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:24.704241 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e86ec-3cdb-4881-a5c5-572373138c11 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:24.704518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e86ec-3cdb-4881-a5c5-572373138c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:24.705106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e86ec-3cdb-4881-a5c5-572373138c11 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:24.705730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e86ec-3cdb-4881-a5c5-572373138c11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:24.707165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e86ec-3cdb-4881-a5c5-572373138c11 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:24.710216 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c8513-5e37-4c95-a36b-79fd0eedbca4 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:24.710623 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c8513-5e37-4c95-a36b-79fd0eedbca4 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:24.711019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c8513-5e37-4c95-a36b-79fd0eedbca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:24.716233 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 446/913] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Aug 23 07:00:24 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:00:24.737598 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-9a9c8513-5e37-4c95-a36b-79fd0eedbca4 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:24.739175 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9a9c8513-5e37-4c95-a36b-79fd0eedbca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:24.753961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:24.827518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 468/914] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:25.092690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e86ec-3cdb-4881-a5c5-572373138c11 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:25.138960 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:25.197554 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e86ec-3cdb-4881-a5c5-572373138c11 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:25.232579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76476cf4-a3a5-4b00-90be-a131710cc643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:25.233209 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76476cf4-a3a5-4b00-90be-a131710cc643 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:25.233483 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76476cf4-a3a5-4b00-90be-a131710cc643 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:25.233749 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76476cf4-a3a5-4b00-90be-a131710cc643 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:25.234335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76476cf4-a3a5-4b00-90be-a131710cc643 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:25.235735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76476cf4-a3a5-4b00-90be-a131710cc643 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:25.244800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-76476cf4-a3a5-4b00-90be-a131710cc643 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:00:25.244800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:25.244800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:25.244800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:25.244800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:25.244800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:25.244800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:25.244800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:25.244800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:00:25.244800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:25.244800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:25.244800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:25.244800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:25.244800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:25.244800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:25.244800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:25.244800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:25.245833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:25.245833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:25.245833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:25.245833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:25.245833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:25.245833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:25.245833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:25.245833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:25.245833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:25.245833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:25.245833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:25.245833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:25.245833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:25.245833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:25.245833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:25.245833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:25.246673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:25.246673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:25.246673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:25.246673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:25.246673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:25.246673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:25.246673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:25.246673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:25.246673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:25.246673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:25.246673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:25.246673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:25.246673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:00:25.246673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:25.246673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 447/915] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 07:00:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:25.252257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c8513-5e37-4c95-a36b-79fd0eedbca4 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:25.365431 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c8513-5e37-4c95-a36b-79fd0eedbca4 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:25.400912 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c59ff2-8df1-4b4c-9e7a-c1f22cf87f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:25.401845 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c59ff2-8df1-4b4c-9e7a-c1f22cf87f6c None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:25.402344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c59ff2-8df1-4b4c-9e7a-c1f22cf87f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:25.402813 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c59ff2-8df1-4b4c-9e7a-c1f22cf87f6c None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:25.403749 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c59ff2-8df1-4b4c-9e7a-c1f22cf87f6c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:25.404248 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c59ff2-8df1-4b4c-9e7a-c1f22cf87f6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:25.405819 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c59ff2-8df1-4b4c-9e7a-c1f22cf87f6c None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:25.411078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 469/916] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 07:00:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:25.416716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76476cf4-a3a5-4b00-90be-a131710cc643 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:25.518797 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76476cf4-a3a5-4b00-90be-a131710cc643 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:25.528049 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:25.553238 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf0c06-b503-4c03-825f-61dfd9206ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:25.554027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf0c06-b503-4c03-825f-61dfd9206ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:25.554298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf0c06-b503-4c03-825f-61dfd9206ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:25.554561 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf0c06-b503-4c03-825f-61dfd9206ce4 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:25.555199 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf0c06-b503-4c03-825f-61dfd9206ce4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:25.555522 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf0c06-b503-4c03-825f-61dfd9206ce4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:25.556909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf0c06-b503-4c03-825f-61dfd9206ce4 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:25.562281 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 448/917] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Aug 23 07:00:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:25.567754 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c59ff2-8df1-4b4c-9e7a-c1f22cf87f6c None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:25.581436 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:25.673250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c59ff2-8df1-4b4c-9e7a-c1f22cf87f6c None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:25.709098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5cd68-56be-4f99-88db-c21d6474f1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:25.709723 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5cd68-56be-4f99-88db-c21d6474f1b8 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:25.710005 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5cd68-56be-4f99-88db-c21d6474f1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:25.710281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5cd68-56be-4f99-88db-c21d6474f1b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:25.710957 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5cd68-56be-4f99-88db-c21d6474f1b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:25.712521 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5cd68-56be-4f99-88db-c21d6474f1b8 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:25.743691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 470/918] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:00:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:00:26.142440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c6212-8a21-4594-86bf-7678cdcce887 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:26.143489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c6212-8a21-4594-86bf-7678cdcce887 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:26.143993 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c6212-8a21-4594-86bf-7678cdcce887 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:26.145784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 449/919] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:26.151531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaedab9-d9e3-49aa-a6d9-30abfc78cbfa None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:26.152074 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaedab9-d9e3-49aa-a6d9-30abfc78cbfa None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:26.152567 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaedab9-d9e3-49aa-a6d9-30abfc78cbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:26.171180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-ccaedab9-d9e3-49aa-a6d9-30abfc78cbfa None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:26.172760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ccaedab9-d9e3-49aa-a6d9-30abfc78cbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:26.241946 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:26.263121 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 471/920] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:26.529802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c6212-8a21-4594-86bf-7678cdcce887 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:26.638088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c6212-8a21-4594-86bf-7678cdcce887 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:26.652724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e318a-0776-476b-ae4f-a4c46b30a0e6 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:26.653344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e318a-0776-476b-ae4f-a4c46b30a0e6 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:26.653814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e318a-0776-476b-ae4f-a4c46b30a0e6 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:26.656491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 472/921] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:26.661965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcff807-afe4-43a5-88fb-725c37de230b None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:26.662678 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcff807-afe4-43a5-88fb-725c37de230b None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:26.663254 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcff807-afe4-43a5-88fb-725c37de230b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:26.673336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4118841-c3f7-41c9-ad8b-9de81498d412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:26.673964 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4118841-c3f7-41c9-ad8b-9de81498d412 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:26.674238 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4118841-c3f7-41c9-ad8b-9de81498d412 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:26.674505 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4118841-c3f7-41c9-ad8b-9de81498d412 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:26.675087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4118841-c3f7-41c9-ad8b-9de81498d412 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:26.676630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4118841-c3f7-41c9-ad8b-9de81498d412 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:26.684432 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-9dcff807-afe4-43a5-88fb-725c37de230b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:26.685571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-c4118841-c3f7-41c9-ad8b-9de81498d412 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:00:26.685571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:26.685571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:26.685571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:26.685571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:26.685571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:26.685571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:26.685571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:26.685571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:00:26.685571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:26.685571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:26.685571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:26.685571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:26.685571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:26.685571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:26.685571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:26.685571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:26.686535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:26.686535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:26.686535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:26.686535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:26.686535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:26.686535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:26.686535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:26.686535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:26.686535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:26.686535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:26.686535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:26.686535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:26.686535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:26.686535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:26.686535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:26.686535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:26.687366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:26.687366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:26.687366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:26.687366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:26.687366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:26.687366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:26.687366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:26.687366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:26.687366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:26.687366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:26.687366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:26.687366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:26.687366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:00:26.687366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:26.688148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9dcff807-afe4-43a5-88fb-725c37de230b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:26.688220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 450/922] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 23 07:00:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:26.688220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686eb4f-5fec-4346-9afe-84bc1f5dc146 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:26.688930 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686eb4f-5fec-4346-9afe-84bc1f5dc146 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:26.689210 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686eb4f-5fec-4346-9afe-84bc1f5dc146 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:26.691444 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 451/923] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:26.692284 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686eb4f-5fec-4346-9afe-84bc1f5dc146 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:26.729673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:26.763351 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 473/924] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:26.764969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53bdde-bb94-45fb-a15b-42f27bab03ab None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:26.765452 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53bdde-bb94-45fb-a15b-42f27bab03ab None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:26.765946 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53bdde-bb94-45fb-a15b-42f27bab03ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:26.786580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-9a53bdde-bb94-45fb-a15b-42f27bab03ab None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:26.788266 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9a53bdde-bb94-45fb-a15b-42f27bab03ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:26.810372 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686eb4f-5fec-4346-9afe-84bc1f5dc146 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:26.848264 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e435e-a8cd-4d68-842b-27daba3a06be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:26.849291 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e435e-a8cd-4d68-842b-27daba3a06be None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:26.849734 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e435e-a8cd-4d68-842b-27daba3a06be None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:26.850187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e435e-a8cd-4d68-842b-27daba3a06be None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:26.851088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e435e-a8cd-4d68-842b-27daba3a06be None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:26.851637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e435e-a8cd-4d68-842b-27daba3a06be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:26.853328 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e435e-a8cd-4d68-842b-27daba3a06be None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:26.859365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 452/925] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 23 07:00:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:26.864962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e435e-a8cd-4d68-842b-27daba3a06be None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:26.866693 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 474/926] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:26.963054 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:26.967983 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e435e-a8cd-4d68-842b-27daba3a06be None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:27.003443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b307b76-b806-423f-a7a7-e582f0547ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:27.004109 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b307b76-b806-423f-a7a7-e582f0547ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:27.004347 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b307b76-b806-423f-a7a7-e582f0547ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:27.004620 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b307b76-b806-423f-a7a7-e582f0547ef2 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:27.005244 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b307b76-b806-423f-a7a7-e582f0547ef2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:27.005541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b307b76-b806-423f-a7a7-e582f0547ef2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:27.006936 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b307b76-b806-423f-a7a7-e582f0547ef2 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:27.012269 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 453/927] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 23 07:00:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:27.018171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53bdde-bb94-45fb-a15b-42f27bab03ab None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:27.031830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b307b76-b806-423f-a7a7-e582f0547ef2 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:27.072653 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:27.120949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53bdde-bb94-45fb-a15b-42f27bab03ab None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:27.139441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b307b76-b806-423f-a7a7-e582f0547ef2 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:27.157241 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd464de5-8523-4659-9b1e-f96953ce56f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:27.158149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd464de5-8523-4659-9b1e-f96953ce56f6 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:27.158616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd464de5-8523-4659-9b1e-f96953ce56f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:27.158884 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd464de5-8523-4659-9b1e-f96953ce56f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:27.159617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd464de5-8523-4659-9b1e-f96953ce56f6 None admin] RBAC: Adding query filter params (interface=public, service_id=44af491b74094481b9e8b364bb8c85f4, region_id=RegionOne) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:27.159922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd464de5-8523-4659-9b1e-f96953ce56f6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:27.161633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd464de5-8523-4659-9b1e-f96953ce56f6 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:27.171014 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 475/928] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Aug 23 07:00:27 2022] GET /identity/v3/endpoints?service_id=44af491b74094481b9e8b364bb8c85f4&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:27.171014 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd464de5-8523-4659-9b1e-f96953ce56f6 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:27.181372 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0336090-dcc8-4b03-a9cb-6304bfe6f0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:27.182145 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0336090-dcc8-4b03-a9cb-6304bfe6f0dc None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:27.182316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0336090-dcc8-4b03-a9cb-6304bfe6f0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:27.182561 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0336090-dcc8-4b03-a9cb-6304bfe6f0dc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:27.193560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0336090-dcc8-4b03-a9cb-6304bfe6f0dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:00:27.198460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-a0336090-dcc8-4b03-a9cb-6304bfe6f0dc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:00:27.198460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:27.198460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:27.198460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:27.198460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:27.198460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:27.198460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:27.198460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:27.198460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:27.198460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:27.198460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:27.198460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:27.198460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:00:27.198460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:00:27.198460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:00:27.198460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:00:27.199389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:00:27.199389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:00:27.199389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:00:27.199389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:00:27.199389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:27.199389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:27.199389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:27.199389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:27.199389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:27.199389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:27.199389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:27.199389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:27.199389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:27.199389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:27.199389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:27.199389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:27.200825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:27.200825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:27.200825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:27.200825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:27.200825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:00:27.200825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:00:27.200825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:00:27.200825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:00:27.200825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:00:27.200825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:00:27.200825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:00:27.200825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:27.200825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 454/929] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Aug 23 07:00:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:27.200825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0336090-dcc8-4b03-a9cb-6304bfe6f0dc None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:27.226926 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:27.279927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd464de5-8523-4659-9b1e-f96953ce56f6 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:27.296219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:27.309066 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0336090-dcc8-4b03-a9cb-6304bfe6f0dc None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:27.313746 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edc21c-b237-4ddc-8d03-8690eeffb66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:27.314376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edc21c-b237-4ddc-8d03-8690eeffb66d None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:27.314648 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edc21c-b237-4ddc-8d03-8690eeffb66d None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:27.314914 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edc21c-b237-4ddc-8d03-8690eeffb66d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:27.315518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edc21c-b237-4ddc-8d03-8690eeffb66d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:27.316886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edc21c-b237-4ddc-8d03-8690eeffb66d None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:27.326302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-a5edc21c-b237-4ddc-8d03-8690eeffb66d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:00:27.326302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:27.326302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:27.326302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:27.326302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:27.326302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:27.326302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:27.326302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:27.326302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:00:27.326302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:27.326302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:27.326302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:27.326302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:27.326302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:27.326302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:27.326302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:27.327250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:27.327250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:27.327250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:27.327250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:27.327250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:27.327250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:27.327250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:27.327250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:27.327250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:27.327250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:27.327250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:27.327250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:27.327250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:27.327250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:27.327250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:27.327250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:27.327250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:27.328172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:27.328172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:27.328172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:27.328172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:27.328172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:27.328172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:27.328172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:27.328172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:27.328172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:27.328172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:27.328172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:27.328172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:27.328172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:00:27.328172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:27.328172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 476/930] 10.222.0.17 () {56 vars in 1132 bytes} [Tue Aug 23 07:00:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:27.328953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edc21c-b237-4ddc-8d03-8690eeffb66d None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:27.352174 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54968e-7f1c-4368-ae27-5c6243bb5e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:27.352812 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54968e-7f1c-4368-ae27-5c6243bb5e97 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:27.353093 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54968e-7f1c-4368-ae27-5c6243bb5e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:27.353363 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54968e-7f1c-4368-ae27-5c6243bb5e97 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:27.353972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54968e-7f1c-4368-ae27-5c6243bb5e97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:27.355637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54968e-7f1c-4368-ae27-5c6243bb5e97 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:27.367284 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 455/931] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Aug 23 07:00:27 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:00:27.368268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54968e-7f1c-4368-ae27-5c6243bb5e97 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:27.466936 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edc21c-b237-4ddc-8d03-8690eeffb66d None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:27.499490 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:27.509304 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378129c9-0e10-43c4-81a2-95e9840ab58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:27.509832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378129c9-0e10-43c4-81a2-95e9840ab58f None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:27.510187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378129c9-0e10-43c4-81a2-95e9840ab58f None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:27.510619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378129c9-0e10-43c4-81a2-95e9840ab58f None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:27.511287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378129c9-0e10-43c4-81a2-95e9840ab58f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:27.511681 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378129c9-0e10-43c4-81a2-95e9840ab58f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:27.513437 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378129c9-0e10-43c4-81a2-95e9840ab58f None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:27.516915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54968e-7f1c-4368-ae27-5c6243bb5e97 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:27.527787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 477/932] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Aug 23 07:00:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:27.528732 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378129c9-0e10-43c4-81a2-95e9840ab58f None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:27.566985 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013c163-1f45-4343-ac52-740fe951bf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:27.568270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013c163-1f45-4343-ac52-740fe951bf1d None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:27.568788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013c163-1f45-4343-ac52-740fe951bf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:27.569350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013c163-1f45-4343-ac52-740fe951bf1d None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:27.570959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013c163-1f45-4343-ac52-740fe951bf1d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:27.571426 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013c163-1f45-4343-ac52-740fe951bf1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:27.573133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013c163-1f45-4343-ac52-740fe951bf1d None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:27.580703 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 456/933] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 23 07:00:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:27.586183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:27.586832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013c163-1f45-4343-ac52-740fe951bf1d None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:27.655782 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378129c9-0e10-43c4-81a2-95e9840ab58f None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:27.698750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013c163-1f45-4343-ac52-740fe951bf1d None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:27.699489 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b38ab-7e9b-4c38-9c0d-d9c8fa04c0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:27.700530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b38ab-7e9b-4c38-9c0d-d9c8fa04c0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:27.701132 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b38ab-7e9b-4c38-9c0d-d9c8fa04c0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:27.701609 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b38ab-7e9b-4c38-9c0d-d9c8fa04c0f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:27.715297 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b38ab-7e9b-4c38-9c0d-d9c8fa04c0f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:27.717220 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b38ab-7e9b-4c38-9c0d-d9c8fa04c0f5 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:27.733193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-4b3b38ab-7e9b-4c38-9c0d-d9c8fa04c0f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:27.733193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:27.733193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:27.733193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:27.733193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:27.733193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:27.733193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:27.733193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:27.733193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:27.733193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:27.733193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:27.733193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:27.733193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:27.733193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:27.733193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:27.733193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:27.734303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:27.734303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:27.734303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:27.734303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:27.734303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:27.734303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:27.734303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:27.734303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:27.734303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:27.734303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:27.734303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:27.734303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:27.734303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:27.734303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:27.734303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:27.734303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:27.734303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:27.735340 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:27.735340 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:27.735340 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:27.735340 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 478/934] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:27.736158 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4983d-af66-404f-afca-d0814366bbb8 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:27.736158 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4983d-af66-404f-afca-d0814366bbb8 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:27.736158 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4983d-af66-404f-afca-d0814366bbb8 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:27.737410 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 479/935] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:27.740887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3404f4d-6e02-43e7-94df-8fe692583d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:27.741734 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3404f4d-6e02-43e7-94df-8fe692583d99 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:27.741991 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3404f4d-6e02-43e7-94df-8fe692583d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:27.742335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3404f4d-6e02-43e7-94df-8fe692583d99 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:27.742624 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4983d-af66-404f-afca-d0814366bbb8 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:27.743157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3404f4d-6e02-43e7-94df-8fe692583d99 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:27.743609 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3404f4d-6e02-43e7-94df-8fe692583d99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:27.745025 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3404f4d-6e02-43e7-94df-8fe692583d99 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:27.755492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 457/936] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 23 07:00:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:27.755492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1804a-75b1-43ab-aa47-3747bc1c7ee4 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:27.755492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1804a-75b1-43ab-aa47-3747bc1c7ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:27.755492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1804a-75b1-43ab-aa47-3747bc1c7ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:27.772334 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-a3d1804a-75b1-43ab-aa47-3747bc1c7ee4 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:27.773758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-a3d1804a-75b1-43ab-aa47-3747bc1c7ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:27.786223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:27.847072 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 458/937] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:27.847865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1804a-75b1-43ab-aa47-3747bc1c7ee4 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:27.872550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4983d-af66-404f-afca-d0814366bbb8 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:27.875709 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:27.912029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd3151-78ed-4db7-87e9-cdda6319e400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:27.912887 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd3151-78ed-4db7-87e9-cdda6319e400 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:27.913397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd3151-78ed-4db7-87e9-cdda6319e400 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:27.913770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd3151-78ed-4db7-87e9-cdda6319e400 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:27.914811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd3151-78ed-4db7-87e9-cdda6319e400 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:27.915198 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd3151-78ed-4db7-87e9-cdda6319e400 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:27.917837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd3151-78ed-4db7-87e9-cdda6319e400 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:27.941035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 480/938] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:27.942936 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd3151-78ed-4db7-87e9-cdda6319e400 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:27.971813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1804a-75b1-43ab-aa47-3747bc1c7ee4 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:28.018923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec9a47-5e70-4cea-b7b6-06cb5ec03748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:28.019579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec9a47-5e70-4cea-b7b6-06cb5ec03748 None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:28.019905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec9a47-5e70-4cea-b7b6-06cb5ec03748 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:28.020182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec9a47-5e70-4cea-b7b6-06cb5ec03748 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:28.020884 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec9a47-5e70-4cea-b7b6-06cb5ec03748 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:28.022806 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec9a47-5e70-4cea-b7b6-06cb5ec03748 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:28.065325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 459/939] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:00:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:00:28.075732 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd3151-78ed-4db7-87e9-cdda6319e400 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:28.121487 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec9a47-5e70-4cea-b7b6-06cb5ec03748 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:28.124143 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48e08d-6bfb-4307-aab6-26ed57207d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:28.124789 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48e08d-6bfb-4307-aab6-26ed57207d35 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:28.125059 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48e08d-6bfb-4307-aab6-26ed57207d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:28.125371 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48e08d-6bfb-4307-aab6-26ed57207d35 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:28.127382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:28.134244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48e08d-6bfb-4307-aab6-26ed57207d35 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:28.136030 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48e08d-6bfb-4307-aab6-26ed57207d35 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:28.148940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-1e48e08d-6bfb-4307-aab6-26ed57207d35 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:00:28.148940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:28.148940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:28.148940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:28.148940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:28.148940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:28.148940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:28.148940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:28.148940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:28.148940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:28.148940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:28.148940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:28.148940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:00:28.148940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:00:28.148940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:00:28.148940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:00:28.150009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:28.150009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:28.150009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:00:28.150009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:28.150009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:00:28.150009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:28.150009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:28.150009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:28.150009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:28.150009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:28.150009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:28.150009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:28.150009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:28.150009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:28.150009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:28.150009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:28.150009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:28.150009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:28.150965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:28.150965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:28.150965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:00:28.150965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:00:28.150965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:00:28.150965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:00:28.150965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:00:28.150965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:00:28.150965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:00:28.150965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:28.155024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 481/940] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 23 07:00:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:28.156568 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48e08d-6bfb-4307-aab6-26ed57207d35 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:28.170501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:28.258165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec9a47-5e70-4cea-b7b6-06cb5ec03748 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:28.276941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48e08d-6bfb-4307-aab6-26ed57207d35 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:28.299045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b796cf6-f7b0-40d4-8a90-0b6ec43b801a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:28.300112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b796cf6-f7b0-40d4-8a90-0b6ec43b801a None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:28.300398 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b796cf6-f7b0-40d4-8a90-0b6ec43b801a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:28.300674 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b796cf6-f7b0-40d4-8a90-0b6ec43b801a None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:28.301332 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b796cf6-f7b0-40d4-8a90-0b6ec43b801a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:28.303282 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b796cf6-f7b0-40d4-8a90-0b6ec43b801a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:28.306144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: WARNING keystone.server.flask.common [None req-4b796cf6-f7b0-40d4-8a90-0b6ec43b801a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 23 07:00:28.320563 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb07955-3ab8-4e8c-bba6-11e8941e361a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:28.321927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb07955-3ab8-4e8c-bba6-11e8941e361a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:28.322485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb07955-3ab8-4e8c-bba6-11e8941e361a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:28.322955 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb07955-3ab8-4e8c-bba6-11e8941e361a None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:28.323937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb07955-3ab8-4e8c-bba6-11e8941e361a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:28.324445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb07955-3ab8-4e8c-bba6-11e8941e361a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:28.326086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb07955-3ab8-4e8c-bba6-11e8941e361a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:28.331476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.sql.core [None req-4b796cf6-f7b0-40d4-8a90-0b6ec43b801a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 07:00:28.331476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 23 07:00:28.331476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [parameters: {'id': 'c773be5d1d1a4dce91b090e451fa2470', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 23 07:00:28.331476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81714) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 07:00:28.338728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-4b796cf6-f7b0-40d4-8a90-0b6ec43b801a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c773be5d1d1a4dce91b090e451fa2470.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c773be5d1d1a4dce91b090e451fa2470. Aug 23 07:00:28.338728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:28.338728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 07:00:28.338728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 07:00:28.338728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 07:00:28.338728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 07:00:28.338728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 07:00:28.338728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 07:00:28.338728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 07:00:28.338728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application conn.query(q) Aug 23 07:00:28.338728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 07:00:28.338728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 07:00:28.338728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 07:00:28.339578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application result.read() Aug 23 07:00:28.339578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 07:00:28.339578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 07:00:28.339578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 07:00:28.339578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 07:00:28.339578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 07:00:28.339578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 07:00:28.339578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 07:00:28.339578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 07:00:28.339578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 07:00:28.339578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:28.339578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 07:00:28.339578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:28.339578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:28.339578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 07:00:28.339578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 07:00:28.339578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 23 07:00:28.339578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 23 07:00:28.340948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 07:00:28.340948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application next(self.gen) Aug 23 07:00:28.340948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 07:00:28.340948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application yield resource Aug 23 07:00:28.340948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 07:00:28.340948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application next(self.gen) Aug 23 07:00:28.340948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 07:00:28.340948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 07:00:28.340948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 07:00:28.340948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 07:00:28.340948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 07:00:28.340948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise self.value Aug 23 07:00:28.340948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 07:00:28.340948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 07:00:28.340948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 07:00:28.340948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application session.commit() Aug 23 07:00:28.340948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 07:00:28.342088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 07:00:28.342088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 07:00:28.342088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 07:00:28.342088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 07:00:28.342088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self.session.flush() Aug 23 07:00:28.342088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 07:00:28.342088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 07:00:28.342088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 07:00:28.342088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 07:00:28.342088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 07:00:28.342088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application compat.raise_( Aug 23 07:00:28.342088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 07:00:28.342088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception Aug 23 07:00:28.342088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 07:00:28.342088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 07:00:28.342088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 07:00:28.342088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 07:00:28.342972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 07:00:28.342972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 07:00:28.342972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 07:00:28.342972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 07:00:28.342972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 23 07:00:28.342972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 23 07:00:28.342972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 07:00:28.342972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 07:00:28.342972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 07:00:28.342972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 07:00:28.342972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 07:00:28.342972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 07:00:28.342972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 07:00:28.342972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 07:00:28.342972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 07:00:28.343789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 07:00:28.343789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 07:00:28.343789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception Aug 23 07:00:28.343789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 07:00:28.343789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 07:00:28.343789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 07:00:28.343789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 07:00:28.343789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 07:00:28.343789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 07:00:28.343789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 07:00:28.343789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application conn.query(q) Aug 23 07:00:28.343789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 07:00:28.343789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 07:00:28.343789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 07:00:28.343789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application result.read() Aug 23 07:00:28.343789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 07:00:28.343789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 07:00:28.344639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 07:00:28.344639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 07:00:28.344639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 07:00:28.344639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 07:00:28.344639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 07:00:28.344639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 07:00:28.344639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 07:00:28.344639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 23 07:00:28.344639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [parameters: {'id': 'c773be5d1d1a4dce91b090e451fa2470', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 23 07:00:28.344639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 07:00:28.344639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:28.344639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:28.344639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:28.344639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:28.344639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 23 07:00:28.347336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 23 07:00:28.347336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 07:00:28.347336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 07:00:28.347336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 23 07:00:28.347336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:28.347336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:28.347336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:28.347336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:28.347336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:28.347336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:28.347336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:28.347336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:28.347336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:28.347336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:28.347336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:28.347336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:28.347336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:28.349518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:28.349518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 23 07:00:28.349518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 23 07:00:28.349518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:28.349518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:28.349518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 23 07:00:28.349518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 23 07:00:28.349518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c773be5d1d1a4dce91b090e451fa2470. Aug 23 07:00:28.349518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:28.349518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 460/941] 10.222.0.17 () {60 vars in 1140 bytes} [Tue Aug 23 07:00:28 2022] POST /identity/v3/projects => generated 262 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:28.349518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b796cf6-f7b0-40d4-8a90-0b6ec43b801a None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:28.363379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 482/942] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Aug 23 07:00:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:28.365041 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb07955-3ab8-4e8c-bba6-11e8941e361a None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:28.412175 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:28.436954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:28.464445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b796cf6-f7b0-40d4-8a90-0b6ec43b801a None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:28.473772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb07955-3ab8-4e8c-bba6-11e8941e361a None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:28.502155 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f4745-a80c-4f27-be43-9e102e35ced0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:28.502804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f4745-a80c-4f27-be43-9e102e35ced0 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:28.503088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f4745-a80c-4f27-be43-9e102e35ced0 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:28.503365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f4745-a80c-4f27-be43-9e102e35ced0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:28.511120 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4854d3-18bf-4e18-a77f-3b4161bf6e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:28.512106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f4745-a80c-4f27-be43-9e102e35ced0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:28.512429 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4854d3-18bf-4e18-a77f-3b4161bf6e67 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:28.512898 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4854d3-18bf-4e18-a77f-3b4161bf6e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:28.513389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4854d3-18bf-4e18-a77f-3b4161bf6e67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:28.513833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f4745-a80c-4f27-be43-9e102e35ced0 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:28.526224 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4854d3-18bf-4e18-a77f-3b4161bf6e67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:28.527342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-fa2f4745-a80c-4f27-be43-9e102e35ced0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:28.527342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:28.527342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:28.527342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:28.527342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:28.527342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:28.527342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:28.527342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:28.527342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:28.527342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:28.527342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:28.527342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:28.527342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:00:28.527342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:00:28.527342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:00:28.528235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:00:28.528235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:28.528235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:28.528235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:28.528235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:28.528235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:28.528235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:28.528235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:28.528235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:28.528235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:28.528235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:28.528235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:28.528235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:28.528235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:28.528235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:28.528235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:28.528235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:28.529134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:00:28.529134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:00:28.529134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:00:28.529134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:00:28.529134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:00:28.529134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:00:28.529134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:28.529134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:28.530967 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4854d3-18bf-4e18-a77f-3b4161bf6e67 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:28.533612 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 461/943] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:28.534558 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f4745-a80c-4f27-be43-9e102e35ced0 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:28.549619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-bc4854d3-18bf-4e18-a77f-3b4161bf6e67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:28.549619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:28.549619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:28.549619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:28.549619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:28.549619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:28.549619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:28.549619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:28.549619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:28.549619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:28.549619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:28.549619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:28.549619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:28.549619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:28.549619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:28.549619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:28.550640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:28.550640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:28.550640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:28.550640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:28.550640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:28.550640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:28.550640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:28.550640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:28.550640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:28.550640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:28.550640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:28.550640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:28.550640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:28.550640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:28.550640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:28.550640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:28.550640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:28.551534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:28.551534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:28.551534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:28.558241 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 483/944] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:28.559907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4854d3-18bf-4e18-a77f-3b4161bf6e67 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:28.644000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f4745-a80c-4f27-be43-9e102e35ced0 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:28.664366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:28.666607 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4854d3-18bf-4e18-a77f-3b4161bf6e67 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:28.684667 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c1052-ce41-4652-982a-9135da86ce97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:28.685294 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c1052-ce41-4652-982a-9135da86ce97 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:28.685576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c1052-ce41-4652-982a-9135da86ce97 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:28.685850 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c1052-ce41-4652-982a-9135da86ce97 None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:28.686522 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c1052-ce41-4652-982a-9135da86ce97 None admin] RBAC: Adding query filter params (name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:28.686824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c1052-ce41-4652-982a-9135da86ce97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:28.688298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c1052-ce41-4652-982a-9135da86ce97 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:28.689168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:28.704205 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e09d87-ba7b-46fe-b5b2-fd36c5003b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:28.704945 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e09d87-ba7b-46fe-b5b2-fd36c5003b91 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:28.705625 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e09d87-ba7b-46fe-b5b2-fd36c5003b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:28.705992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e09d87-ba7b-46fe-b5b2-fd36c5003b91 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:28.706749 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e09d87-ba7b-46fe-b5b2-fd36c5003b91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:28.707143 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e09d87-ba7b-46fe-b5b2-fd36c5003b91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:28.708647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e09d87-ba7b-46fe-b5b2-fd36c5003b91 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:28.723934 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 462/946] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 23 07:00:28 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:28.725989 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 484/946] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:28.727468 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c1052-ce41-4652-982a-9135da86ce97 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:28.844666 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c1052-ce41-4652-982a-9135da86ce97 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:28.882169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33570ee-9a75-40fa-af4f-4c2490bdc05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:28.882814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33570ee-9a75-40fa-af4f-4c2490bdc05c None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:28.883094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33570ee-9a75-40fa-af4f-4c2490bdc05c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:28.883369 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33570ee-9a75-40fa-af4f-4c2490bdc05c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:28.890374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33570ee-9a75-40fa-af4f-4c2490bdc05c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:28.891954 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33570ee-9a75-40fa-af4f-4c2490bdc05c None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:28.903453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-d33570ee-9a75-40fa-af4f-4c2490bdc05c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:28.903453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:28.903453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:28.903453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:28.903453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:28.903453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:28.903453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:28.903453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:28.903453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:28.903453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:28.903453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:28.903453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:28.903453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:00:28.903453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:00:28.903453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:00:28.904386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:00:28.904386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:28.904386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:28.904386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:28.904386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:28.904386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:28.904386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:28.904386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:28.904386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:28.904386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:28.904386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:28.904386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:28.904386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:28.904386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:28.904386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:28.904386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:28.904386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:28.905420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:00:28.905420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:00:28.905420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:00:28.905420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:00:28.905420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:00:28.905420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:00:28.905420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:28.905420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:28.905420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 463/947] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:28.912509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e09d87-ba7b-46fe-b5b2-fd36c5003b91 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:29.018265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e09d87-ba7b-46fe-b5b2-fd36c5003b91 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:29.053498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6915a1-3698-4195-8410-c99c1a620cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:29.054558 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6915a1-3698-4195-8410-c99c1a620cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:29.055036 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6915a1-3698-4195-8410-c99c1a620cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:29.055557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6915a1-3698-4195-8410-c99c1a620cf0 None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:29.056448 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6915a1-3698-4195-8410-c99c1a620cf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:29.056963 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6915a1-3698-4195-8410-c99c1a620cf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:29.058578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6915a1-3698-4195-8410-c99c1a620cf0 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:29.075659 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 485/948] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:00:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:29.077954 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33570ee-9a75-40fa-af4f-4c2490bdc05c None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:29.121984 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:29.184221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33570ee-9a75-40fa-af4f-4c2490bdc05c None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:29.219047 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b9b72-5cc7-4bd6-bed4-04c4f35a8660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:29.220116 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b9b72-5cc7-4bd6-bed4-04c4f35a8660 None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:29.220604 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b9b72-5cc7-4bd6-bed4-04c4f35a8660 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:29.221065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b9b72-5cc7-4bd6-bed4-04c4f35a8660 None admin] PATH_INFO: `/v3/projects/351a6777525c4207826d7df0d3768e99/users/b3f1ae88e29f4172a7db7393d517a893/roles/6d1af19135484d85af7accf2fdeb3cab` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:29.223602 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b9b72-5cc7-4bd6-bed4-04c4f35a8660 None admin] RBAC: Authorizing `identity:create_grant(project_id=351a6777525c4207826d7df0d3768e99, user_id=b3f1ae88e29f4172a7db7393d517a893, role_id=6d1af19135484d85af7accf2fdeb3cab)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:29.225416 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b9b72-5cc7-4bd6-bed4-04c4f35a8660 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:29.246463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 464/949] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 07:00:29 2022] PUT /identity/v3/projects/351a6777525c4207826d7df0d3768e99/users/b3f1ae88e29f4172a7db7393d517a893/roles/6d1af19135484d85af7accf2fdeb3cab => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:00:29.376363 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f397546-ec40-4648-b7a2-f5da888804ed None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:29.377330 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f397546-ec40-4648-b7a2-f5da888804ed None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:29.377856 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f397546-ec40-4648-b7a2-f5da888804ed None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:29.379786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 486/950] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:29.385457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaabc3-b1c4-4b5b-bf2a-b6f57d5bc722 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:29.385896 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaabc3-b1c4-4b5b-bf2a-b6f57d5bc722 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:29.385990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaabc3-b1c4-4b5b-bf2a-b6f57d5bc722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:29.407641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-83eaabc3-b1c4-4b5b-bf2a-b6f57d5bc722 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:29.409163 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-83eaabc3-b1c4-4b5b-bf2a-b6f57d5bc722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:29.480883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 465/951] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:29.746512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f397546-ec40-4648-b7a2-f5da888804ed None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:29.792629 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:29.852777 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f397546-ec40-4648-b7a2-f5da888804ed None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:29.887171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87083652-6803-4be7-a339-942db8ccc83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:29.887828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87083652-6803-4be7-a339-942db8ccc83b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:29.888110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87083652-6803-4be7-a339-942db8ccc83b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:29.888389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87083652-6803-4be7-a339-942db8ccc83b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:29.888974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87083652-6803-4be7-a339-942db8ccc83b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:29.890352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87083652-6803-4be7-a339-942db8ccc83b None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:29.899171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-87083652-6803-4be7-a339-942db8ccc83b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:00:29.899171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:29.899171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:29.899171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:29.899171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:29.899171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:29.899171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:29.899171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:29.899171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:00:29.899171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:29.899171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:29.899171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:29.899171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:29.899171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:29.899171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:29.899171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:29.899171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:29.900211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:29.900211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:29.900211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:29.900211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:29.900211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:29.900211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:29.900211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:29.900211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:29.900211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:29.900211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:29.900211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:29.900211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:29.900211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:29.900211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:29.900211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:29.900211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:29.901082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:29.901082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:29.901082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:29.901082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:29.901082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:29.901082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:29.901082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:29.901082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:29.901082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:29.901082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:29.901082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:29.901082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:29.901082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:00:29.901082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:29.905208 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 487/952] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 23 07:00:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:29.906643 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaabc3-b1c4-4b5b-bf2a-b6f57d5bc722 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:29.940134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:30.010556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaabc3-b1c4-4b5b-bf2a-b6f57d5bc722 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:30.045356 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2db2c-9ad7-44ee-9ee0-f0fe88855077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:30.045973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2db2c-9ad7-44ee-9ee0-f0fe88855077 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:30.046250 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2db2c-9ad7-44ee-9ee0-f0fe88855077 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:30.046518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2db2c-9ad7-44ee-9ee0-f0fe88855077 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:30.047141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2db2c-9ad7-44ee-9ee0-f0fe88855077 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:30.047462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2db2c-9ad7-44ee-9ee0-f0fe88855077 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:30.049006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2db2c-9ad7-44ee-9ee0-f0fe88855077 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:30.058780 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 466/953] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 23 07:00:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:30.060168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87083652-6803-4be7-a339-942db8ccc83b None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:30.082748 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b6b2d-48ee-4f02-9fb3-19e38e159c66 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:30.083050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b6b2d-48ee-4f02-9fb3-19e38e159c66 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:30.083287 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b6b2d-48ee-4f02-9fb3-19e38e159c66 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:30.084897 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 467/954] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:00:30.089752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f2758-2baf-4430-99b9-eca804994de0 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:30.090035 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f2758-2baf-4430-99b9-eca804994de0 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:30.090309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f2758-2baf-4430-99b9-eca804994de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:30.108635 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-355f2758-2baf-4430-99b9-eca804994de0 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:30.109838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-355f2758-2baf-4430-99b9-eca804994de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:30.183814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87083652-6803-4be7-a339-942db8ccc83b None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:30.187073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:30.192683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 468/955] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:30.229388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfc59f-4fe6-4459-8020-07d38bd578d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:30.229997 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfc59f-4fe6-4459-8020-07d38bd578d4 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:30.230299 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfc59f-4fe6-4459-8020-07d38bd578d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:30.230579 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfc59f-4fe6-4459-8020-07d38bd578d4 None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:30.231262 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfc59f-4fe6-4459-8020-07d38bd578d4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:30.231594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfc59f-4fe6-4459-8020-07d38bd578d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:30.233073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfc59f-4fe6-4459-8020-07d38bd578d4 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:30.238826 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 488/956] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 23 07:00:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:30.244793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f2758-2baf-4430-99b9-eca804994de0 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:30.280762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:30.348103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f2758-2baf-4430-99b9-eca804994de0 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:30.385704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6dc84a-dca1-41d3-a062-31d993c40864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:30.386505 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dc84a-dca1-41d3-a062-31d993c40864 None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:30.386700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dc84a-dca1-41d3-a062-31d993c40864 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:30.386866 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dc84a-dca1-41d3-a062-31d993c40864 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:30.387500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6dc84a-dca1-41d3-a062-31d993c40864 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:30.388881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6dc84a-dca1-41d3-a062-31d993c40864 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:30.415559 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 469/957] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:00:30 2022] POST /identity/v3/endpoints => generated 349 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:00:30.501476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfc59f-4fe6-4459-8020-07d38bd578d4 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:30.613530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfc59f-4fe6-4459-8020-07d38bd578d4 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:30.654883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa8eb4-5ff9-406b-bd7d-83d86fbdf1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:30.656150 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa8eb4-5ff9-406b-bd7d-83d86fbdf1bc None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:30.656150 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa8eb4-5ff9-406b-bd7d-83d86fbdf1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:30.656150 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa8eb4-5ff9-406b-bd7d-83d86fbdf1bc None admin] PATH_INFO: `/v3/services/compute` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:30.656612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa8eb4-5ff9-406b-bd7d-83d86fbdf1bc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:30.658000 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa8eb4-5ff9-406b-bd7d-83d86fbdf1bc None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:30.667010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-60fa8eb4-5ff9-406b-bd7d-83d86fbdf1bc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:00:30.667010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:30.667010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:30.667010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:30.667010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:30.667010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:30.667010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:30.667010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:30.667010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:00:30.667010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:30.667010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:30.667010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:30.667010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:30.667010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:30.667010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:30.667010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:30.667010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:30.669541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:30.669541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:30.669541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:30.669541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:30.669541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:30.669541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:30.669541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:30.669541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:30.669541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:30.669541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:30.669541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:30.669541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:30.669541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:30.669541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:30.669541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:30.669541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:30.671511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:30.671511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:30.671511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:30.671511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:30.671511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:30.671511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:30.671511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:30.671511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:30.671511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:30.671511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:30.671511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:30.671511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:30.671511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:00:30.671511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:30.671511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 489/958] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:30.685266 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6dc84a-dca1-41d3-a062-31d993c40864 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:30.768121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:30.792051 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6dc84a-dca1-41d3-a062-31d993c40864 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:30.817926 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c1973-95f9-41ce-a85e-19d3a0c07562 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:30.818290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c1973-95f9-41ce-a85e-19d3a0c07562 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:30.818468 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c1973-95f9-41ce-a85e-19d3a0c07562 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:30.820135 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 490/959] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:30.825137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ad491-e279-479f-866d-1f2fb4a113d6 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:30.825418 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ad491-e279-479f-866d-1f2fb4a113d6 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:30.825691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ad491-e279-479f-866d-1f2fb4a113d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:30.827063 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8e537-7caf-4cbf-8ebc-b0b4ca3b5896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:30.827708 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8e537-7caf-4cbf-8ebc-b0b4ca3b5896 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:30.827982 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8e537-7caf-4cbf-8ebc-b0b4ca3b5896 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:30.828252 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8e537-7caf-4cbf-8ebc-b0b4ca3b5896 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:30.828890 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8e537-7caf-4cbf-8ebc-b0b4ca3b5896 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:30.829191 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8e537-7caf-4cbf-8ebc-b0b4ca3b5896 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:30.830912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8e537-7caf-4cbf-8ebc-b0b4ca3b5896 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:30.836404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 470/960] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 23 07:00:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:30.844315 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-2e7ad491-e279-479f-866d-1f2fb4a113d6 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:30.844489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8e537-7caf-4cbf-8ebc-b0b4ca3b5896 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:30.845498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-2e7ad491-e279-479f-866d-1f2fb4a113d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:30.876841 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:30.923677 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 491/961] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:30.961344 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8e537-7caf-4cbf-8ebc-b0b4ca3b5896 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:30.995883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39210cbb-5f14-4453-aa18-cb6bb6127001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:30.996494 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39210cbb-5f14-4453-aa18-cb6bb6127001 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:30.996832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39210cbb-5f14-4453-aa18-cb6bb6127001 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:30.997112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39210cbb-5f14-4453-aa18-cb6bb6127001 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:30.997785 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39210cbb-5f14-4453-aa18-cb6bb6127001 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:30.998094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39210cbb-5f14-4453-aa18-cb6bb6127001 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:30.999504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39210cbb-5f14-4453-aa18-cb6bb6127001 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:31.004503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 471/962] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 23 07:00:30 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:31.180901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ad491-e279-479f-866d-1f2fb4a113d6 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:31.229011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4eadd9-bbd8-40ad-9a2f-b67c2a53cfc6 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:31.229011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4eadd9-bbd8-40ad-9a2f-b67c2a53cfc6 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:31.229011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4eadd9-bbd8-40ad-9a2f-b67c2a53cfc6 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:31.229011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 472/963] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:00:31.229011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6eecb-3969-47cb-9809-f34a533e347e None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:31.229011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6eecb-3969-47cb-9809-f34a533e347e None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:31.229828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6eecb-3969-47cb-9809-f34a533e347e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:31.247808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:31.254983 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-a5d6eecb-3969-47cb-9809-f34a533e347e None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:31.256525 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-a5d6eecb-3969-47cb-9809-f34a533e347e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:31.284557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ad491-e279-479f-866d-1f2fb4a113d6 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:31.319790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccec850-5da6-4f09-ab3f-14363d240405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:31.320487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccec850-5da6-4f09-ab3f-14363d240405 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:31.320829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccec850-5da6-4f09-ab3f-14363d240405 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:31.321167 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccec850-5da6-4f09-ab3f-14363d240405 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:31.321963 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccec850-5da6-4f09-ab3f-14363d240405 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:31.323632 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccec850-5da6-4f09-ab3f-14363d240405 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:31.326308 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: WARNING keystone.server.flask.common [None req-dccec850-5da6-4f09-ab3f-14363d240405 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 23 07:00:31.339960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.sql.core [None req-dccec850-5da6-4f09-ab3f-14363d240405 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 07:00:31.339960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 23 07:00:31.339960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [parameters: {'user_id': 'b0e15a436d774e94a69802a96c5f0289', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 23 07:00:31.339960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81715) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 07:00:31.351513 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 473/964] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:31.356867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-dccec850-5da6-4f09-ab3f-14363d240405 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 23 07:00:31.356867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:31.356867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 07:00:31.356867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 07:00:31.356867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 07:00:31.356867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 07:00:31.356867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 07:00:31.356867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 07:00:31.356867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 07:00:31.356867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application conn.query(q) Aug 23 07:00:31.356867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 07:00:31.356867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 07:00:31.356867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 07:00:31.356867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application result.read() Aug 23 07:00:31.356867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 07:00:31.357758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 07:00:31.357758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 07:00:31.357758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 07:00:31.357758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 07:00:31.357758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 07:00:31.357758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 07:00:31.357758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 07:00:31.357758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 07:00:31.357758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:31.357758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 07:00:31.357758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:31.357758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:31.357758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 07:00:31.357758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 07:00:31.357758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 23 07:00:31.357758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 23 07:00:31.357758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 07:00:31.357758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application next(self.gen) Aug 23 07:00:31.358708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 07:00:31.358708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application yield resource Aug 23 07:00:31.358708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 07:00:31.358708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application next(self.gen) Aug 23 07:00:31.358708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 07:00:31.358708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 07:00:31.358708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 07:00:31.358708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 07:00:31.358708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 07:00:31.358708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise self.value Aug 23 07:00:31.358708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 07:00:31.358708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 07:00:31.358708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 07:00:31.358708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application session.commit() Aug 23 07:00:31.358708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 07:00:31.358708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 07:00:31.358708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 07:00:31.359638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 07:00:31.359638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 07:00:31.359638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self.session.flush() Aug 23 07:00:31.359638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 07:00:31.359638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 07:00:31.359638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 07:00:31.359638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 07:00:31.359638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 07:00:31.359638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application compat.raise_( Aug 23 07:00:31.359638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 07:00:31.359638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception Aug 23 07:00:31.359638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 07:00:31.359638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 07:00:31.359638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 07:00:31.359638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 07:00:31.359638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 07:00:31.359638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 07:00:31.360522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 07:00:31.360522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 07:00:31.360522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 23 07:00:31.360522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 23 07:00:31.360522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 07:00:31.360522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 07:00:31.360522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 07:00:31.360522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 07:00:31.360522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 07:00:31.360522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 07:00:31.360522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 07:00:31.360522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 07:00:31.360522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 07:00:31.360522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 07:00:31.360522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 07:00:31.360522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception Aug 23 07:00:31.361382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 07:00:31.361382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 07:00:31.361382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 07:00:31.361382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 07:00:31.361382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 07:00:31.361382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 07:00:31.361382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 07:00:31.361382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application conn.query(q) Aug 23 07:00:31.361382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 07:00:31.361382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 07:00:31.361382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 07:00:31.361382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application result.read() Aug 23 07:00:31.361382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 07:00:31.361382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 07:00:31.361382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 07:00:31.361382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 07:00:31.362218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 07:00:31.362218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 07:00:31.362218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 07:00:31.362218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 07:00:31.362218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 07:00:31.362218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 23 07:00:31.362218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b0e15a436d774e94a69802a96c5f0289', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 23 07:00:31.362218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 07:00:31.362218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:31.362218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:31.362218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:31.362218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:31.362218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:31.362218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:31.362218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:31.363006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:31.363006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:31.363006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:31.363006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:31.363006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:31.363006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:31.363006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:31.363006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 23 07:00:31.363006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 23 07:00:31.363006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:31.363006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:31.363006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:00:31.363006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:31.363006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:00:31.363006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:31.363006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 23 07:00:31.363006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 23 07:00:31.364491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 23 07:00:31.364491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 23 07:00:31.364491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 07:00:31.364491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 07:00:31.364491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 23 07:00:31.364491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:31.364491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 492/965] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Aug 23 07:00:31 2022] POST /identity/v3/users => generated 160 bytes in 178 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:31.367703 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6eecb-3969-47cb-9809-f34a533e347e None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:31.469916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6eecb-3969-47cb-9809-f34a533e347e None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:31.506109 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68fc1d-9425-40c5-baee-69bbb7f35527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:31.507846 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68fc1d-9425-40c5-baee-69bbb7f35527 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:31.507846 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68fc1d-9425-40c5-baee-69bbb7f35527 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:31.507846 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68fc1d-9425-40c5-baee-69bbb7f35527 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:31.519436 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e68fc1d-9425-40c5-baee-69bbb7f35527 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:31.519650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:31.520655 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e68fc1d-9425-40c5-baee-69bbb7f35527 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:31.534560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-1e68fc1d-9425-40c5-baee-69bbb7f35527 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 07:00:31.534560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:31.534560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:31.534560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:31.534560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:31.534560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:31.534560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:31.534560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:31.534560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:31.534560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:31.534560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:31.534560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:31.534560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:00:31.534560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:00:31.534560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:00:31.534560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:00:31.536959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:31.536959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:31.536959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:00:31.536959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:31.536959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:00:31.536959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:31.536959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:31.536959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:31.536959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:31.536959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:31.536959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:31.536959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:31.536959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:31.536959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:31.536959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:31.536959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:31.536959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:31.536959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:31.541795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:31.541795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:31.541795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:00:31.541795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:00:31.541795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:00:31.541795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:00:31.541795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:00:31.541795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:00:31.541795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 07:00:31.541795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:31.541795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 474/966] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 07:00:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:31.548915 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccec850-5da6-4f09-ab3f-14363d240405 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:31.567995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:31.645262 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68fc1d-9425-40c5-baee-69bbb7f35527 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:31.654542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccec850-5da6-4f09-ab3f-14363d240405 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:31.696243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef75131-1316-47fc-b518-906d982889c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:31.696860 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef75131-1316-47fc-b518-906d982889c8 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:31.697135 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef75131-1316-47fc-b518-906d982889c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:31.697407 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef75131-1316-47fc-b518-906d982889c8 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:31.698111 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef75131-1316-47fc-b518-906d982889c8 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:31.698408 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef75131-1316-47fc-b518-906d982889c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:31.700614 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef75131-1316-47fc-b518-906d982889c8 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:31.743397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 493/967] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Aug 23 07:00:31 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:31.769512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68fc1d-9425-40c5-baee-69bbb7f35527 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:31.807951 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0d6b8-9950-4de2-a037-6b715572faec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:31.808992 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0d6b8-9950-4de2-a037-6b715572faec None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:31.809474 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0d6b8-9950-4de2-a037-6b715572faec None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:31.809971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0d6b8-9950-4de2-a037-6b715572faec None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:31.822705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0d6b8-9950-4de2-a037-6b715572faec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:00:31.829710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-37c0d6b8-9950-4de2-a037-6b715572faec None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:00:31.829710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:31.829710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:31.829710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:31.829710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:31.829710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:31.829710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:31.829710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:31.829710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:31.829710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:31.829710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:31.829710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:31.829710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:00:31.829710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:00:31.829710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:00:31.829710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:00:31.831053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:00:31.831053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:00:31.831053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:00:31.831053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:00:31.831053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:31.831053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:31.831053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:31.831053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:31.831053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:31.831053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:31.831053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:31.831053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:31.831053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:31.831053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:31.831053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:31.831053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:31.831974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:31.831974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:31.831974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:31.831974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:31.831974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:00:31.831974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:00:31.831974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:00:31.831974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:00:31.831974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:00:31.831974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:00:31.831974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:00:31.831974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:31.834924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 475/968] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Aug 23 07:00:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:31.841354 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef75131-1316-47fc-b518-906d982889c8 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:31.930005 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:31.954064 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef75131-1316-47fc-b518-906d982889c8 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:31.991203 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4e4f2-a035-4b65-bc4d-4137f96ea71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:31.991960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4e4f2-a035-4b65-bc4d-4137f96ea71d None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:31.992242 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4e4f2-a035-4b65-bc4d-4137f96ea71d None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:31.992516 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4e4f2-a035-4b65-bc4d-4137f96ea71d None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:31.993189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4e4f2-a035-4b65-bc4d-4137f96ea71d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:31.993542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4e4f2-a035-4b65-bc4d-4137f96ea71d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:31.994895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4e4f2-a035-4b65-bc4d-4137f96ea71d None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:32.004878 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 494/969] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Aug 23 07:00:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:32.010600 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0d6b8-9950-4de2-a037-6b715572faec None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:32.073873 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:32.120700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0d6b8-9950-4de2-a037-6b715572faec None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:32.154729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96411319-4eb2-42af-8075-54bff5f7c2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:32.155354 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96411319-4eb2-42af-8075-54bff5f7c2de None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:32.155652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96411319-4eb2-42af-8075-54bff5f7c2de None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:32.155925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96411319-4eb2-42af-8075-54bff5f7c2de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:32.167252 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96411319-4eb2-42af-8075-54bff5f7c2de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:32.168751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96411319-4eb2-42af-8075-54bff5f7c2de None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:32.183154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-96411319-4eb2-42af-8075-54bff5f7c2de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:32.183154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:32.183154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:32.183154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:32.183154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:32.183154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:32.183154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:32.183154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:32.183154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:32.183154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:32.183154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:32.183154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:32.183154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:32.183154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:32.183154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:32.183154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:32.184161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:32.184161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:32.184161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:32.184161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:32.184161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:32.184161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:32.184161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:32.184161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:32.184161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:32.184161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:32.184161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:32.184161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:32.184161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:32.184161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:32.184161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:32.184161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:32.184161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:32.185075 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:32.185075 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:32.185075 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:32.185075 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 476/970] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:32.191357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4e4f2-a035-4b65-bc4d-4137f96ea71d None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:32.294850 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4e4f2-a035-4b65-bc4d-4137f96ea71d None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:32.327882 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12675ba-cc32-4b03-928b-18e708634e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:32.328480 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12675ba-cc32-4b03-928b-18e708634e40 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:32.328750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12675ba-cc32-4b03-928b-18e708634e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:32.329012 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12675ba-cc32-4b03-928b-18e708634e40 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:32.329641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12675ba-cc32-4b03-928b-18e708634e40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:32.329933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12675ba-cc32-4b03-928b-18e708634e40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:32.331618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12675ba-cc32-4b03-928b-18e708634e40 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:32.340649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:32.349084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 495/971] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:32.350540 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96411319-4eb2-42af-8075-54bff5f7c2de None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:32.415491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8273134-041d-48db-95f4-caf9fbc1b92d None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:32.415780 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8273134-041d-48db-95f4-caf9fbc1b92d None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:32.416057 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8273134-041d-48db-95f4-caf9fbc1b92d None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:32.417566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 496/972] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:32.422373 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3bdf4-860f-4795-add0-5abf6b9ac0ba None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:32.422653 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3bdf4-860f-4795-add0-5abf6b9ac0ba None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:32.422927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3bdf4-860f-4795-add0-5abf6b9ac0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:32.441229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-b1a3bdf4-860f-4795-add0-5abf6b9ac0ba None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:32.442961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b1a3bdf4-860f-4795-add0-5abf6b9ac0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:32.459505 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96411319-4eb2-42af-8075-54bff5f7c2de None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:32.469584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:32.496901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28978d3-a058-4115-ada5-2b7fb61e6926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:32.497539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28978d3-a058-4115-ada5-2b7fb61e6926 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:32.497795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28978d3-a058-4115-ada5-2b7fb61e6926 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:32.498057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28978d3-a058-4115-ada5-2b7fb61e6926 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:32.506102 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28978d3-a058-4115-ada5-2b7fb61e6926 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:32.507775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28978d3-a058-4115-ada5-2b7fb61e6926 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:32.516193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 497/973] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:32.518775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-d28978d3-a058-4115-ada5-2b7fb61e6926 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:00:32.518775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:32.518775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:32.518775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:32.518775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:32.518775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:32.518775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:32.518775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:32.518775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:32.518775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:32.518775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:32.518775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:32.518775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:00:32.518775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:00:32.518775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:00:32.518775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:00:32.519716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:32.519716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:32.519716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:00:32.519716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:32.519716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:00:32.519716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:32.519716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:32.519716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:32.519716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:32.519716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:32.519716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:32.519716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:32.519716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:32.519716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:32.519716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:32.519716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:32.519716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:32.519716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:32.521083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:32.521083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:32.521083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:00:32.521083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:00:32.521083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:00:32.521083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:00:32.521083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:00:32.521083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:00:32.521083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:00:32.521083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:32.521083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 477/974] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 23 07:00:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:32.525658 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3bdf4-860f-4795-add0-5abf6b9ac0ba None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:32.628663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3bdf4-860f-4795-add0-5abf6b9ac0ba None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:32.664485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:32.671290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2f245-72b9-4f85-aba3-a071aab9eba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:32.672104 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2f245-72b9-4f85-aba3-a071aab9eba8 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:32.672528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2f245-72b9-4f85-aba3-a071aab9eba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:32.672942 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2f245-72b9-4f85-aba3-a071aab9eba8 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:32.673840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2f245-72b9-4f85-aba3-a071aab9eba8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:32.674346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2f245-72b9-4f85-aba3-a071aab9eba8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:32.675969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2f245-72b9-4f85-aba3-a071aab9eba8 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:32.694523 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 498/975] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Aug 23 07:00:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:32.700283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28978d3-a058-4115-ada5-2b7fb61e6926 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:32.780757 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2f245-72b9-4f85-aba3-a071aab9eba8 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:32.809666 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28978d3-a058-4115-ada5-2b7fb61e6926 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:32.844617 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b728aee-2877-4d28-86a1-eda99386fba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:32.845228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b728aee-2877-4d28-86a1-eda99386fba5 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:32.845512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b728aee-2877-4d28-86a1-eda99386fba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:32.845783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b728aee-2877-4d28-86a1-eda99386fba5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:32.880472 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:32.901098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2f245-72b9-4f85-aba3-a071aab9eba8 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:32.917553 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b728aee-2877-4d28-86a1-eda99386fba5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:32.919029 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b728aee-2877-4d28-86a1-eda99386fba5 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:32.939377 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1ef85-b548-4b78-aba3-270c0f4c9168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:32.940036 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1ef85-b548-4b78-aba3-270c0f4c9168 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:32.940312 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1ef85-b548-4b78-aba3-270c0f4c9168 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:32.940591 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1ef85-b548-4b78-aba3-270c0f4c9168 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:32.941179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1ef85-b548-4b78-aba3-270c0f4c9168 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:32.942547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1ef85-b548-4b78-aba3-270c0f4c9168 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:32.951375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-6ab1ef85-b548-4b78-aba3-270c0f4c9168 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:00:32.951375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:32.951375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:32.951375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:32.951375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:32.951375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:32.951375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:32.951375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:32.951375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:00:32.951375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:32.951375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:32.951375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:32.951375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:32.951375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:32.951375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:32.951375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:32.951375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:32.952557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:32.952557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:32.952557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:32.952557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:32.952557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:32.952557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:32.952557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:32.952557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:32.952557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:32.952557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:32.952557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:32.952557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:32.952557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:32.952557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:32.952557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:32.952557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:32.953881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:32.953881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:32.953881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:32.953881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:32.953881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:32.953881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:32.953881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:32.953881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:32.953881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:32.953881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:32.953881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:32.953881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:32.953881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:00:32.953881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:32.953881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 499/976] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 23 07:00:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:32.954871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3605ebe-90ad-4ea1-a283-64409d8336c3 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:32.954871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3605ebe-90ad-4ea1-a283-64409d8336c3 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:32.954871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3605ebe-90ad-4ea1-a283-64409d8336c3 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:32.955976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 500/977] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:32.960424 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3605ebe-90ad-4ea1-a283-64409d8336c3 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:33.016761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:33.023854 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 478/978] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:00:32 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:00:33.029566 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a650af-6165-4ff2-9099-2b0b5652bb95 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:33.029854 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a650af-6165-4ff2-9099-2b0b5652bb95 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:33.030124 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a650af-6165-4ff2-9099-2b0b5652bb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:33.048045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-37a650af-6165-4ff2-9099-2b0b5652bb95 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:33.049246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-37a650af-6165-4ff2-9099-2b0b5652bb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:33.065669 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3605ebe-90ad-4ea1-a283-64409d8336c3 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:33.105320 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ed911-ddbc-4ac6-948e-943e2b01f913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:33.105924 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ed911-ddbc-4ac6-948e-943e2b01f913 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:33.106199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ed911-ddbc-4ac6-948e-943e2b01f913 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:33.106469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ed911-ddbc-4ac6-948e-943e2b01f913 None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:33.107093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ed911-ddbc-4ac6-948e-943e2b01f913 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:33.107390 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ed911-ddbc-4ac6-948e-943e2b01f913 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:33.108798 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ed911-ddbc-4ac6-948e-943e2b01f913 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:33.114120 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 501/979] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 23 07:00:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:33.114869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ed911-ddbc-4ac6-948e-943e2b01f913 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:33.116617 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:33.127590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 479/980] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:33.127590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a650af-6165-4ff2-9099-2b0b5652bb95 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:33.227338 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ed911-ddbc-4ac6-948e-943e2b01f913 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:33.233834 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a650af-6165-4ff2-9099-2b0b5652bb95 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:33.244644 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:33.263915 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7a84b-b4f0-4014-97fc-5a5368544d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:33.264547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7a84b-b4f0-4014-97fc-5a5368544d37 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:33.264824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7a84b-b4f0-4014-97fc-5a5368544d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:33.265097 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7a84b-b4f0-4014-97fc-5a5368544d37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:33.270827 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7562c4-bd58-48d8-bafa-c3d0c9d65ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:33.271464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7562c4-bd58-48d8-bafa-c3d0c9d65ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:33.271744 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7562c4-bd58-48d8-bafa-c3d0c9d65ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:33.272048 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7562c4-bd58-48d8-bafa-c3d0c9d65ca3 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:33.273350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7562c4-bd58-48d8-bafa-c3d0c9d65ca3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:33.273656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7562c4-bd58-48d8-bafa-c3d0c9d65ca3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:33.275053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7562c4-bd58-48d8-bafa-c3d0c9d65ca3 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:33.276343 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7a84b-b4f0-4014-97fc-5a5368544d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:33.277784 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7a84b-b4f0-4014-97fc-5a5368544d37 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:33.280316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 480/981] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 23 07:00:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:33.286027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7562c4-bd58-48d8-bafa-c3d0c9d65ca3 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:33.292600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-f5f7a84b-b4f0-4014-97fc-5a5368544d37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:33.292600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:33.292600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:33.292600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:33.292600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:33.292600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:33.292600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:33.292600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:33.292600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:33.292600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:33.292600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:33.292600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:33.292600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:33.292600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:33.292600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:33.292600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:33.293539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:33.293539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:33.293539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:33.293539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:33.293539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:33.293539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:33.293539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:33.293539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:33.293539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:33.293539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:33.293539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:33.293539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:33.293539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:33.293539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:33.293539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:33.293539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:33.293539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:33.294434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:33.294434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:33.294434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:33.294434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 502/982] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:33.299369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7a84b-b4f0-4014-97fc-5a5368544d37 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:33.353190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:33.393839 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7562c4-bd58-48d8-bafa-c3d0c9d65ca3 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:33.408082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7a84b-b4f0-4014-97fc-5a5368544d37 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:33.427816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247fc45-bfd5-4ff6-8fe6-fc11d488b68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:33.428564 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247fc45-bfd5-4ff6-8fe6-fc11d488b68c None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:33.428848 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247fc45-bfd5-4ff6-8fe6-fc11d488b68c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:33.429125 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247fc45-bfd5-4ff6-8fe6-fc11d488b68c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:33.429838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f247fc45-bfd5-4ff6-8fe6-fc11d488b68c None admin] RBAC: Adding query filter params (interface=admin, service_id=44af491b74094481b9e8b364bb8c85f4, region_id=RegionOne) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:33.430150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f247fc45-bfd5-4ff6-8fe6-fc11d488b68c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:33.431645 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f247fc45-bfd5-4ff6-8fe6-fc11d488b68c None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:33.436773 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 481/983] 10.222.0.17 () {56 vars in 1249 bytes} [Tue Aug 23 07:00:33 2022] GET /identity/v3/endpoints?service_id=44af491b74094481b9e8b364bb8c85f4&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:33.437526 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247fc45-bfd5-4ff6-8fe6-fc11d488b68c None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:33.445269 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019338a-4f55-4af1-b485-29cbf470951b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:33.445891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019338a-4f55-4af1-b485-29cbf470951b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:33.446210 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019338a-4f55-4af1-b485-29cbf470951b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:33.446477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019338a-4f55-4af1-b485-29cbf470951b None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:33.447123 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1019338a-4f55-4af1-b485-29cbf470951b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:33.447441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1019338a-4f55-4af1-b485-29cbf470951b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:33.448815 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1019338a-4f55-4af1-b485-29cbf470951b None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:33.461389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 503/984] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:33.462197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019338a-4f55-4af1-b485-29cbf470951b None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:33.489488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:33.540860 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247fc45-bfd5-4ff6-8fe6-fc11d488b68c None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:33.562353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019338a-4f55-4af1-b485-29cbf470951b None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:33.566095 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:33.575841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de27ea-b5e8-420b-b74f-b38c0e1a6ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:33.576448 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de27ea-b5e8-420b-b74f-b38c0e1a6ed2 None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:33.576728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de27ea-b5e8-420b-b74f-b38c0e1a6ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:33.576997 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de27ea-b5e8-420b-b74f-b38c0e1a6ed2 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:33.577666 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de27ea-b5e8-420b-b74f-b38c0e1a6ed2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:33.579046 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de27ea-b5e8-420b-b74f-b38c0e1a6ed2 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:33.595254 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 482/985] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 23 07:00:33 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:00:33.600416 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de27ea-b5e8-420b-b74f-b38c0e1a6ed2 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:33.609030 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6541be9-65e6-4b27-b340-df46b8263ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:33.609660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6541be9-65e6-4b27-b340-df46b8263ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:33.609942 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6541be9-65e6-4b27-b340-df46b8263ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:33.610214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6541be9-65e6-4b27-b340-df46b8263ec8 None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:33.610868 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6541be9-65e6-4b27-b340-df46b8263ec8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:33.612339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6541be9-65e6-4b27-b340-df46b8263ec8 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:33.620118 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 504/986] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Aug 23 07:00:33 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:00:33.740510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de27ea-b5e8-420b-b74f-b38c0e1a6ed2 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:33.781947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f6b61-d7c0-4e66-ae26-8601edee1505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:33.782450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f6b61-d7c0-4e66-ae26-8601edee1505 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:33.782685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f6b61-d7c0-4e66-ae26-8601edee1505 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:33.782989 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f6b61-d7c0-4e66-ae26-8601edee1505 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:33.794326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f6b61-d7c0-4e66-ae26-8601edee1505 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:33.795932 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f6b61-d7c0-4e66-ae26-8601edee1505 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:33.808999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-5a8f6b61-d7c0-4e66-ae26-8601edee1505 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:33.808999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:33.808999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:33.808999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:33.808999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:33.808999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:33.808999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:33.808999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:33.808999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:33.808999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:33.808999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:33.808999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:33.808999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:00:33.808999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:00:33.808999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:00:33.809949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:00:33.809949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:33.809949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:33.809949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:33.809949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:33.809949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:33.809949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:33.809949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:33.809949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:33.809949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:33.809949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:33.809949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:33.809949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:33.809949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:33.809949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:33.809949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:33.809949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:33.810992 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:00:33.810992 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:00:33.810992 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:00:33.810992 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:00:33.810992 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:00:33.810992 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:00:33.810992 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:33.810992 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:33.814996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 483/987] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:33.816447 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6541be9-65e6-4b27-b340-df46b8263ec8 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:33.918347 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:33.935439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6541be9-65e6-4b27-b340-df46b8263ec8 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:33.974160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60963320-05a8-4e2e-82c5-06a77f32977a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:33.974773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60963320-05a8-4e2e-82c5-06a77f32977a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:33.975052 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60963320-05a8-4e2e-82c5-06a77f32977a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:33.975398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60963320-05a8-4e2e-82c5-06a77f32977a None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:33.976133 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60963320-05a8-4e2e-82c5-06a77f32977a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:33.976441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60963320-05a8-4e2e-82c5-06a77f32977a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:33.977962 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60963320-05a8-4e2e-82c5-06a77f32977a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:33.990672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 505/988] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:00:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:33.997298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f6b61-d7c0-4e66-ae26-8601edee1505 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:34.050164 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:34.112208 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f6b61-d7c0-4e66-ae26-8601edee1505 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:34.146427 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9192a-3c7c-481b-80b8-90947f22458b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:34.147032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9192a-3c7c-481b-80b8-90947f22458b None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:34.147353 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9192a-3c7c-481b-80b8-90947f22458b None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:34.147672 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9192a-3c7c-481b-80b8-90947f22458b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:34.148389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9192a-3c7c-481b-80b8-90947f22458b None admin] RBAC: Adding query filter params (role.id=6d1af19135484d85af7accf2fdeb3cab, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=b3f1ae88e29f4172a7db7393d517a893) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:34.148688 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9192a-3c7c-481b-80b8-90947f22458b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:34.150116 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9192a-3c7c-481b-80b8-90947f22458b None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:34.163449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 484/989] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:00:33 2022] GET /identity/v3/role_assignments?user.id=b3f1ae88e29f4172a7db7393d517a893&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=6d1af19135484d85af7accf2fdeb3cab => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:35.542641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad10ab-a56b-455f-b727-2fc32bae511f None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:35.543187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad10ab-a56b-455f-b727-2fc32bae511f None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:35.543187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad10ab-a56b-455f-b727-2fc32bae511f None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:35.544736 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 506/990] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:35.547394 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9b9ea-49d9-49e4-a740-ba825eec598b None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:35.547881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9b9ea-49d9-49e4-a740-ba825eec598b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:35.547961 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9b9ea-49d9-49e4-a740-ba825eec598b None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:35.549436 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 485/991] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:35.549637 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b7e82-25ae-4a4b-bb5e-4a7c1b299aa6 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:35.549913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b7e82-25ae-4a4b-bb5e-4a7c1b299aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:35.550304 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b7e82-25ae-4a4b-bb5e-4a7c1b299aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:35.554634 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce2364-57a1-4dec-87a3-929940728286 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:35.554935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce2364-57a1-4dec-87a3-929940728286 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:35.555212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce2364-57a1-4dec-87a3-929940728286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:35.571618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-346b7e82-25ae-4a4b-bb5e-4a7c1b299aa6 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:35.572818 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-346b7e82-25ae-4a4b-bb5e-4a7c1b299aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:35.574648 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-58ce2364-57a1-4dec-87a3-929940728286 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:35.575834 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-58ce2364-57a1-4dec-87a3-929940728286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:35.647807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 486/992] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:35.648325 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 507/993] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:35.906181 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce2364-57a1-4dec-87a3-929940728286 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:35.915948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b7e82-25ae-4a4b-bb5e-4a7c1b299aa6 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:35.951752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:35.999938 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:36.016708 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce2364-57a1-4dec-87a3-929940728286 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:36.027545 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b7e82-25ae-4a4b-bb5e-4a7c1b299aa6 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:36.052525 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12e0d6-a7ab-4a50-993c-d8b9aaa3f598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:36.053286 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12e0d6-a7ab-4a50-993c-d8b9aaa3f598 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:36.053610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12e0d6-a7ab-4a50-993c-d8b9aaa3f598 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:36.054195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12e0d6-a7ab-4a50-993c-d8b9aaa3f598 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:36.055048 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12e0d6-a7ab-4a50-993c-d8b9aaa3f598 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:36.056909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12e0d6-a7ab-4a50-993c-d8b9aaa3f598 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:36.064073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a39cc-32dc-44e5-9ce0-5973b6f1a6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:36.064705 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a39cc-32dc-44e5-9ce0-5973b6f1a6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:36.064981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a39cc-32dc-44e5-9ce0-5973b6f1a6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:36.065249 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a39cc-32dc-44e5-9ce0-5973b6f1a6e2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:36.065858 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a39cc-32dc-44e5-9ce0-5973b6f1a6e2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:36.066504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-3d12e0d6-a7ab-4a50-993c-d8b9aaa3f598 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:00:36.066504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:36.066504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:36.066504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:36.066504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:36.066504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:36.066504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:36.066504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:36.066504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:00:36.066504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:36.066504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:36.066504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:36.066504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:36.066504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:36.066504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:36.066504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:36.066504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:36.067829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:36.067829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:36.067829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:36.067829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:36.067829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:36.067829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:36.067829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:36.067829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:36.067829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:36.067829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:36.067829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:36.067829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:36.067829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:36.067829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:36.067829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:36.067829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:36.068784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:36.068784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:36.068784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:36.068784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:36.068784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:36.068784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:36.068784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:36.068784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:36.068784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:36.068784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:36.068784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:36.068784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:36.068784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:00:36.068784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:36.069575 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a39cc-32dc-44e5-9ce0-5973b6f1a6e2 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:36.069648 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 487/994] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:36.073605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12e0d6-a7ab-4a50-993c-d8b9aaa3f598 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:36.080404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-ad4a39cc-32dc-44e5-9ce0-5973b6f1a6e2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:00:36.080404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:36.080404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:36.080404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:36.080404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:36.080404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:36.080404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:36.080404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:36.080404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:00:36.080404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:36.080404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:36.080404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:36.080404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:36.080404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:36.080404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:36.080404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:36.080404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:36.081829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:36.081829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:36.081829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:36.081829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:36.081829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:36.081829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:36.081829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:36.081829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:36.081829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:36.081829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:36.081829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:36.081829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:36.081829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:36.081829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:36.081829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:36.081829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:36.082671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:36.082671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:36.082671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:36.082671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:36.082671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:36.082671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:36.082671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:36.082671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:36.082671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:36.082671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:36.082671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:36.082671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:36.082671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:00:36.082671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:36.082671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 508/995] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Aug 23 07:00:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:36.084982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4e5e0-1edc-4ac2-9e02-eefeb666005c None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:36.084982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4e5e0-1edc-4ac2-9e02-eefeb666005c None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:36.084982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4e5e0-1edc-4ac2-9e02-eefeb666005c None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:36.085911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 509/996] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:36.087826 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4e5e0-1edc-4ac2-9e02-eefeb666005c None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:36.173797 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12e0d6-a7ab-4a50-993c-d8b9aaa3f598 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:36.176387 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:36.189846 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4e5e0-1edc-4ac2-9e02-eefeb666005c None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:36.208128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0020a2-8eb4-424f-95c9-fc2fc5f306dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:36.208771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0020a2-8eb4-424f-95c9-fc2fc5f306dd None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:36.209056 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0020a2-8eb4-424f-95c9-fc2fc5f306dd None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:36.209330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0020a2-8eb4-424f-95c9-fc2fc5f306dd None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:36.210024 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0020a2-8eb4-424f-95c9-fc2fc5f306dd None admin] RBAC: Adding query filter params (name=compute) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:36.210322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0020a2-8eb4-424f-95c9-fc2fc5f306dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:36.211774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0020a2-8eb4-424f-95c9-fc2fc5f306dd None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:36.216527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 488/997] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 23 07:00:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:36.217659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55334cc2-4cae-4760-b713-2e47290ee895 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:36.217945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55334cc2-4cae-4760-b713-2e47290ee895 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:36.218220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55334cc2-4cae-4760-b713-2e47290ee895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:36.226991 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54699b-cfa0-4234-8b0d-cd7dce2bbccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:36.227729 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54699b-cfa0-4234-8b0d-cd7dce2bbccc None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:36.228003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54699b-cfa0-4234-8b0d-cd7dce2bbccc None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:36.228267 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54699b-cfa0-4234-8b0d-cd7dce2bbccc None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:36.228897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54699b-cfa0-4234-8b0d-cd7dce2bbccc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:36.229192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54699b-cfa0-4234-8b0d-cd7dce2bbccc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:36.230548 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54699b-cfa0-4234-8b0d-cd7dce2bbccc None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:36.235294 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 510/998] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 23 07:00:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:36.236122 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54699b-cfa0-4234-8b0d-cd7dce2bbccc None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:36.236229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-55334cc2-4cae-4760-b713-2e47290ee895 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:36.237383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-55334cc2-4cae-4760-b713-2e47290ee895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:36.245650 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:36.307494 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 489/999] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:36.308365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55334cc2-4cae-4760-b713-2e47290ee895 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:36.344226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54699b-cfa0-4234-8b0d-cd7dce2bbccc None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:36.380629 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e77f28-21dc-46af-969c-eb17520b483c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:36.381239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e77f28-21dc-46af-969c-eb17520b483c None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:36.381518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e77f28-21dc-46af-969c-eb17520b483c None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:36.381880 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e77f28-21dc-46af-969c-eb17520b483c None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:36.382512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e77f28-21dc-46af-969c-eb17520b483c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:36.382813 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e77f28-21dc-46af-969c-eb17520b483c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:36.384211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e77f28-21dc-46af-969c-eb17520b483c None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:36.394165 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 511/1000] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 23 07:00:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:36.395263 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e77f28-21dc-46af-969c-eb17520b483c None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:36.412727 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:36.416944 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55334cc2-4cae-4760-b713-2e47290ee895 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:36.452633 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcbb5a-16b1-4733-929a-745037680462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:36.453183 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcbb5a-16b1-4733-929a-745037680462 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:36.453531 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcbb5a-16b1-4733-929a-745037680462 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:36.453816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcbb5a-16b1-4733-929a-745037680462 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:36.454454 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcbb5a-16b1-4733-929a-745037680462 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:36.455269 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcbb5a-16b1-4733-929a-745037680462 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:36.457315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcbb5a-16b1-4733-929a-745037680462 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:36.469096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 490/1001] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Aug 23 07:00:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:36.470309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcbb5a-16b1-4733-929a-745037680462 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:36.472431 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:36.508214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e77f28-21dc-46af-969c-eb17520b483c None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:36.542646 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae76192-508e-4220-b8b9-b7610359f4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:36.543275 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae76192-508e-4220-b8b9-b7610359f4cd None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:36.543924 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae76192-508e-4220-b8b9-b7610359f4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:36.543924 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae76192-508e-4220-b8b9-b7610359f4cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:36.544519 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae76192-508e-4220-b8b9-b7610359f4cd None admin] RBAC: Adding query filter params (interface=public, service_id=92bb44ce77064f7b864f6a02334654c6, region_id=RegionOne) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:36.544820 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae76192-508e-4220-b8b9-b7610359f4cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:36.546168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae76192-508e-4220-b8b9-b7610359f4cd None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:36.551864 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 512/1002] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Aug 23 07:00:36 2022] GET /identity/v3/endpoints?service_id=92bb44ce77064f7b864f6a02334654c6&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:36.557741 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae76192-508e-4220-b8b9-b7610359f4cd None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:36.580302 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcbb5a-16b1-4733-929a-745037680462 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:36.614144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14720be4-0928-4f5e-80d4-82709dba8b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:36.614752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14720be4-0928-4f5e-80d4-82709dba8b7a None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:36.615020 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14720be4-0928-4f5e-80d4-82709dba8b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:36.615292 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14720be4-0928-4f5e-80d4-82709dba8b7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:36.615983 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14720be4-0928-4f5e-80d4-82709dba8b7a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:36.617398 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14720be4-0928-4f5e-80d4-82709dba8b7a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:36.634746 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 491/1003] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:00:36 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:00:36.635598 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14720be4-0928-4f5e-80d4-82709dba8b7a None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:36.667651 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:36.686612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae76192-508e-4220-b8b9-b7610359f4cd None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:36.727686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:36.735877 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee139655-e82a-4915-a2d9-6a86a91e2a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:36.736707 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee139655-e82a-4915-a2d9-6a86a91e2a5e None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:36.736992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee139655-e82a-4915-a2d9-6a86a91e2a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:36.737318 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee139655-e82a-4915-a2d9-6a86a91e2a5e None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:36.738361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee139655-e82a-4915-a2d9-6a86a91e2a5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:36.740109 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee139655-e82a-4915-a2d9-6a86a91e2a5e None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:36.759442 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 513/1004] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Aug 23 07:00:36 2022] GET /identity/v3/services => generated 1727 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:00:36.759666 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14720be4-0928-4f5e-80d4-82709dba8b7a None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:36.801435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30657e-ffcb-421c-85c3-aa586ff803d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:36.802095 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30657e-ffcb-421c-85c3-aa586ff803d4 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:36.802375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30657e-ffcb-421c-85c3-aa586ff803d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:36.802644 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30657e-ffcb-421c-85c3-aa586ff803d4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:36.803252 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30657e-ffcb-421c-85c3-aa586ff803d4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:36.804886 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30657e-ffcb-421c-85c3-aa586ff803d4 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:36.815538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-1d30657e-ffcb-421c-85c3-aa586ff803d4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:00:36.815538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:36.815538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:36.815538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:36.815538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:36.815538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:36.815538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:36.815538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:36.815538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:00:36.815538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:36.815538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:36.815538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:36.815538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:36.815538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:36.815538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:36.815538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:36.815538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:36.816808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:36.816808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:36.816808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:36.816808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:36.816808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:36.816808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:36.816808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:36.816808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:36.816808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:36.816808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:36.816808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:36.816808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:36.816808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:36.816808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:36.816808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:36.816808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:36.817937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:36.817937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:36.817937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:36.817937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:36.817937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:36.817937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:36.817937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:36.817937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:36.817937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:36.817937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:36.817937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:36.817937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:36.817937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:00:36.817937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:36.817937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 492/1005] 10.222.0.17 () {56 vars in 1105 bytes} [Tue Aug 23 07:00:36 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:36.823903 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee139655-e82a-4915-a2d9-6a86a91e2a5e None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:36.934900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee139655-e82a-4915-a2d9-6a86a91e2a5e None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:36.970141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b20e8e-3192-42d8-bd69-2fb2165dfbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:36.971115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b20e8e-3192-42d8-bd69-2fb2165dfbc8 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:36.971648 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b20e8e-3192-42d8-bd69-2fb2165dfbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:36.972127 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b20e8e-3192-42d8-bd69-2fb2165dfbc8 None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:36.972973 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b20e8e-3192-42d8-bd69-2fb2165dfbc8 None admin] RBAC: Adding query filter params (name=image) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:36.973479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b20e8e-3192-42d8-bd69-2fb2165dfbc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:36.975066 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b20e8e-3192-42d8-bd69-2fb2165dfbc8 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:36.980764 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 514/1006] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:36.986181 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30657e-ffcb-421c-85c3-aa586ff803d4 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:37.077414 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:37.094201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30657e-ffcb-421c-85c3-aa586ff803d4 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:37.132678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eefb27-19f3-410b-8fed-a8b33089e8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:37.133301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eefb27-19f3-410b-8fed-a8b33089e8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:37.133579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eefb27-19f3-410b-8fed-a8b33089e8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:37.133829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eefb27-19f3-410b-8fed-a8b33089e8b6 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:37.134492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eefb27-19f3-410b-8fed-a8b33089e8b6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:37.134797 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eefb27-19f3-410b-8fed-a8b33089e8b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:37.136224 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eefb27-19f3-410b-8fed-a8b33089e8b6 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:37.147564 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 493/1007] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:36 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:38.581413 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c099020-8e3b-4f5b-81be-07b63c75abad None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:38.581992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c099020-8e3b-4f5b-81be-07b63c75abad None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:38.581992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c099020-8e3b-4f5b-81be-07b63c75abad None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:38.583618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 515/1008] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:38.588969 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab62e9-a8a8-4aed-888d-ab141c803487 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:38.589422 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab62e9-a8a8-4aed-888d-ab141c803487 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:38.589509 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab62e9-a8a8-4aed-888d-ab141c803487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:38.608204 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-f1ab62e9-a8a8-4aed-888d-ab141c803487 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:38.609390 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f1ab62e9-a8a8-4aed-888d-ab141c803487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:38.654634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4878c-0a38-4800-ae54-d09ae3de3445 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:38.654976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4878c-0a38-4800-ae54-d09ae3de3445 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:38.655189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4878c-0a38-4800-ae54-d09ae3de3445 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:38.656719 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 516/1009] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:38.661745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4287d5-9b7c-4d38-aa78-422513b8afed None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:38.661996 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4287d5-9b7c-4d38-aa78-422513b8afed None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:38.662299 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4287d5-9b7c-4d38-aa78-422513b8afed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:38.677664 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 494/1010] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:38 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:38.683024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-9f4287d5-9b7c-4d38-aa78-422513b8afed None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:38.683024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9f4287d5-9b7c-4d38-aa78-422513b8afed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:38.737268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:38.751802 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 517/1011] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:38 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:38.948969 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab62e9-a8a8-4aed-888d-ab141c803487 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:39.019327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4287d5-9b7c-4d38-aa78-422513b8afed None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:39.039452 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:39.059165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab62e9-a8a8-4aed-888d-ab141c803487 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:39.092942 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b6c29-e120-4546-af6a-a3c49cc8e140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:39.093571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b6c29-e120-4546-af6a-a3c49cc8e140 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:39.093854 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b6c29-e120-4546-af6a-a3c49cc8e140 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:39.094123 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b6c29-e120-4546-af6a-a3c49cc8e140 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:39.095227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b6c29-e120-4546-af6a-a3c49cc8e140 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:39.096709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b6c29-e120-4546-af6a-a3c49cc8e140 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:39.098441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 495/1012] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:39.099378 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e61f3-6c0b-43da-b802-46f01dfaf5d9 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:39.099704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e61f3-6c0b-43da-b802-46f01dfaf5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:39.099973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e61f3-6c0b-43da-b802-46f01dfaf5d9 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:39.103446 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 496/1013] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:39.106200 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e61f3-6c0b-43da-b802-46f01dfaf5d9 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:39.120322 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4287d5-9b7c-4d38-aa78-422513b8afed None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:39.159032 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43faa289-97a8-46fb-be5e-767a1626ebc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:39.159696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43faa289-97a8-46fb-be5e-767a1626ebc3 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:39.159975 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43faa289-97a8-46fb-be5e-767a1626ebc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:39.160248 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43faa289-97a8-46fb-be5e-767a1626ebc3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:39.160831 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43faa289-97a8-46fb-be5e-767a1626ebc3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:39.162211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43faa289-97a8-46fb-be5e-767a1626ebc3 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:39.170992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-43faa289-97a8-46fb-be5e-767a1626ebc3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:00:39.170992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:39.170992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:39.170992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:39.170992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:39.170992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:39.170992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:39.170992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:39.170992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:00:39.170992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:39.170992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:39.170992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:39.170992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:39.170992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:39.170992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:39.170992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:39.170992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:39.172073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:39.172073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:39.172073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:39.172073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:39.172073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:39.172073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:39.172073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:39.172073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:39.172073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:39.172073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:39.172073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:39.172073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:39.172073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:39.172073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:39.172073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:39.172073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:39.172927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:39.172927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:39.172927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:39.172927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:39.172927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:39.172927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:39.172927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:39.172927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:39.172927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:39.172927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:39.172927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:39.172927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:39.172927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:00:39.172927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:39.172927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 518/1014] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:39.173730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429e710-a33a-459c-978c-a95b38753846 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:39.173730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429e710-a33a-459c-978c-a95b38753846 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:39.173730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429e710-a33a-459c-978c-a95b38753846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:39.190864 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-9429e710-a33a-459c-978c-a95b38753846 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:39.192072 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9429e710-a33a-459c-978c-a95b38753846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:39.222664 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e61f3-6c0b-43da-b802-46f01dfaf5d9 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:39.240552 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:39.259066 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d9d76-31e3-442b-8c72-105395b7a3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:39.259703 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d9d76-31e3-442b-8c72-105395b7a3b7 None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:39.259980 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d9d76-31e3-442b-8c72-105395b7a3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:39.260267 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d9d76-31e3-442b-8c72-105395b7a3b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:39.260907 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d9d76-31e3-442b-8c72-105395b7a3b7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:39.261733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 519/1015] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:39 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:39.262348 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d9d76-31e3-442b-8c72-105395b7a3b7 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:39.262560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429e710-a33a-459c-978c-a95b38753846 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:39.284478 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:39.293881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 497/1016] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 23 07:00:39 2022] POST /identity/v3/projects => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:00:39.373613 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429e710-a33a-459c-978c-a95b38753846 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:39.410713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b32cb-0ead-42fe-a387-ce430e1f7d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:39.411327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b32cb-0ead-42fe-a387-ce430e1f7d99 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:39.411629 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b32cb-0ead-42fe-a387-ce430e1f7d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:39.411893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b32cb-0ead-42fe-a387-ce430e1f7d99 None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:39.412527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b32cb-0ead-42fe-a387-ce430e1f7d99 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:39.412824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b32cb-0ead-42fe-a387-ce430e1f7d99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:39.414315 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b32cb-0ead-42fe-a387-ce430e1f7d99 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:39.419917 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 520/1017] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 23 07:00:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:39.425750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d9d76-31e3-442b-8c72-105395b7a3b7 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:39.529448 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b32cb-0ead-42fe-a387-ce430e1f7d99 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:39.534668 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d9d76-31e3-442b-8c72-105395b7a3b7 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:39.573785 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba563b6-3677-4869-ba1e-a0183e4142df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:39.574705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba563b6-3677-4869-ba1e-a0183e4142df None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:39.574988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba563b6-3677-4869-ba1e-a0183e4142df None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:39.575287 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba563b6-3677-4869-ba1e-a0183e4142df None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:39.575922 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba563b6-3677-4869-ba1e-a0183e4142df None admin] RBAC: Adding query filter params (type=compute) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:39.576222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba563b6-3677-4869-ba1e-a0183e4142df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:39.577625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba563b6-3677-4869-ba1e-a0183e4142df None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:39.578458 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:39.583448 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 498/1018] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Aug 23 07:00:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:39.589429 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba563b6-3677-4869-ba1e-a0183e4142df None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:39.644380 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b32cb-0ead-42fe-a387-ce430e1f7d99 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:39.660845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:39.681980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94693e44-ffe4-4fdc-a92b-50b6da6782fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:39.682715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94693e44-ffe4-4fdc-a92b-50b6da6782fd None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:39.683055 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94693e44-ffe4-4fdc-a92b-50b6da6782fd None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:39.683398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94693e44-ffe4-4fdc-a92b-50b6da6782fd None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:39.684139 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94693e44-ffe4-4fdc-a92b-50b6da6782fd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:39.685753 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94693e44-ffe4-4fdc-a92b-50b6da6782fd None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:39.702204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 521/1019] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 23 07:00:39 2022] POST /identity/v3/services => generated 242 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:00:39.708972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba563b6-3677-4869-ba1e-a0183e4142df None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:39.754863 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f979c2-489d-4335-9f67-4bc1ef0e3b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:39.755531 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f979c2-489d-4335-9f67-4bc1ef0e3b34 None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:39.755811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f979c2-489d-4335-9f67-4bc1ef0e3b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:39.756075 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f979c2-489d-4335-9f67-4bc1ef0e3b34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:39.756780 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f979c2-489d-4335-9f67-4bc1ef0e3b34 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:39.758538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f979c2-489d-4335-9f67-4bc1ef0e3b34 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:39.787540 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 499/1020] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:00:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:00:41.244007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016210e-9e08-4ba8-9194-80bcaae1a17a None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:41.244601 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016210e-9e08-4ba8-9194-80bcaae1a17a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:41.244601 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016210e-9e08-4ba8-9194-80bcaae1a17a None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:41.246170 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 522/1021] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:41.251654 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1535f-2e25-4e7a-a3c8-d6245a2f85f7 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:41.252076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1535f-2e25-4e7a-a3c8-d6245a2f85f7 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:41.252261 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1535f-2e25-4e7a-a3c8-d6245a2f85f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:41.270882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-41b1535f-2e25-4e7a-a3c8-d6245a2f85f7 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:41.272133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-41b1535f-2e25-4e7a-a3c8-d6245a2f85f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:41.350951 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 500/1022] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:41.612185 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbaed4-95f6-46e3-9fb6-9eae51edebb2 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:41.612372 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbaed4-95f6-46e3-9fb6-9eae51edebb2 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:41.612614 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbaed4-95f6-46e3-9fb6-9eae51edebb2 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:41.613960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 523/1023] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:41.617647 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1535f-2e25-4e7a-a3c8-d6245a2f85f7 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:41.620605 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2ade7-526f-4100-a82c-ba3eeefdd4bc None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:41.620844 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2ade7-526f-4100-a82c-ba3eeefdd4bc None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:41.621148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2ade7-526f-4100-a82c-ba3eeefdd4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:41.640348 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-67d2ade7-526f-4100-a82c-ba3eeefdd4bc None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:41.641538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-67d2ade7-526f-4100-a82c-ba3eeefdd4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:41.669280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:41.728095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 524/1024] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:41.735382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1535f-2e25-4e7a-a3c8-d6245a2f85f7 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:41.769798 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8cd8a-17f4-411f-999c-17fbe7466c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:41.770658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8cd8a-17f4-411f-999c-17fbe7466c88 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:41.771075 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8cd8a-17f4-411f-999c-17fbe7466c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:41.771546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8cd8a-17f4-411f-999c-17fbe7466c88 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:41.772815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8cd8a-17f4-411f-999c-17fbe7466c88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:41.774536 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8cd8a-17f4-411f-999c-17fbe7466c88 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:41.776639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 501/1025] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:41.782181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2ade7-526f-4100-a82c-ba3eeefdd4bc None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:41.793782 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625350da-ebca-4253-8a42-6a4fbbdc4be8 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:41.794097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625350da-ebca-4253-8a42-6a4fbbdc4be8 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:41.794327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625350da-ebca-4253-8a42-6a4fbbdc4be8 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:41.795996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 502/1026] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:00:41.801392 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2371f2d2-aeb3-47c3-a446-324f06f86e54 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:41.801685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2371f2d2-aeb3-47c3-a446-324f06f86e54 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:41.802179 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2371f2d2-aeb3-47c3-a446-324f06f86e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:41.819799 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-2371f2d2-aeb3-47c3-a446-324f06f86e54 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:41.821623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2371f2d2-aeb3-47c3-a446-324f06f86e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:41.822096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:41.894127 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 503/1027] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:41.900331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2ade7-526f-4100-a82c-ba3eeefdd4bc None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:41.936385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd35da-6ab7-4b66-b223-2fef45a13f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:41.937297 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd35da-6ab7-4b66-b223-2fef45a13f5a None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:41.937772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd35da-6ab7-4b66-b223-2fef45a13f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:41.938236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd35da-6ab7-4b66-b223-2fef45a13f5a None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:41.939102 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd35da-6ab7-4b66-b223-2fef45a13f5a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:41.940816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd35da-6ab7-4b66-b223-2fef45a13f5a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:41.962829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 525/1028] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 23 07:00:41 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:00:42.001034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2371f2d2-aeb3-47c3-a446-324f06f86e54 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:42.053737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:42.122092 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2371f2d2-aeb3-47c3-a446-324f06f86e54 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:42.159111 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c88673-18a7-4904-a500-7154c394558d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:42.159998 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c88673-18a7-4904-a500-7154c394558d None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:42.160434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c88673-18a7-4904-a500-7154c394558d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:42.160846 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c88673-18a7-4904-a500-7154c394558d None admin] PATH_INFO: `/v3/services/image` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:42.161577 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c88673-18a7-4904-a500-7154c394558d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:42.163103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c88673-18a7-4904-a500-7154c394558d None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:42.166904 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd35da-6ab7-4b66-b223-2fef45a13f5a None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:42.181532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-87c88673-18a7-4904-a500-7154c394558d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:00:42.181532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:42.181532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:42.181532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:42.181532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:42.181532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:42.181532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:42.181532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:42.181532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:00:42.181532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:42.181532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:42.181532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:42.181532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:42.181532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:42.181532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:42.181532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:42.181532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:42.182618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:42.182618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:42.182618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:42.182618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:42.182618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:42.182618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:42.182618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:42.182618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:42.182618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:42.182618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:42.182618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:42.182618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:42.182618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:42.182618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:42.182618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:42.182618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:42.183499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:42.183499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:42.183499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:42.183499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:42.183499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:42.183499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:42.183499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:42.183499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:42.183499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:42.183499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:42.183499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:42.183499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:42.183499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:00:42.183499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:42.183499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 504/1029] 10.222.0.17 () {56 vars in 1105 bytes} [Tue Aug 23 07:00:41 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:42.188972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c88673-18a7-4904-a500-7154c394558d None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:42.267546 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:42.278084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd35da-6ab7-4b66-b223-2fef45a13f5a None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:42.297868 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c88673-18a7-4904-a500-7154c394558d None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:42.301089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:42.313684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7218f-c9fa-43d5-9ec7-5337c1d01456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:42.314322 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7218f-c9fa-43d5-9ec7-5337c1d01456 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:42.314601 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7218f-c9fa-43d5-9ec7-5337c1d01456 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:42.314872 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7218f-c9fa-43d5-9ec7-5337c1d01456 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:42.326583 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7218f-c9fa-43d5-9ec7-5337c1d01456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:00:42.331672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-c7b7218f-c9fa-43d5-9ec7-5337c1d01456 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:00:42.331672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:42.331672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:42.331672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:42.331672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:42.331672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:42.331672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:42.331672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:42.331672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:42.331672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:42.331672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:42.331672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:42.331672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:00:42.331672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:00:42.331672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:00:42.331672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:00:42.332814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:00:42.332814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:00:42.332814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:00:42.332814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:00:42.332814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:42.332814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:42.332814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:42.332814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:42.332814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:42.332814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:42.332814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:42.332814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:42.332814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:42.332814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:42.332814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:42.332814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:42.333873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:42.333873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:42.333873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:42.333873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:42.333873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:00:42.333873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:00:42.333873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:00:42.333873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:00:42.333873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:00:42.333873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:00:42.333873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:00:42.333873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:42.338057 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 526/1030] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Aug 23 07:00:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:42.338901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7218f-c9fa-43d5-9ec7-5337c1d01456 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:42.340726 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3b611-7f97-4762-9f7a-01bfffafcc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:42.341358 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3b611-7f97-4762-9f7a-01bfffafcc7d None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:42.341763 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3b611-7f97-4762-9f7a-01bfffafcc7d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:42.341913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3b611-7f97-4762-9f7a-01bfffafcc7d None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:42.343378 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3b611-7f97-4762-9f7a-01bfffafcc7d None admin] RBAC: Adding query filter params (name=image) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:42.343378 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3b611-7f97-4762-9f7a-01bfffafcc7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:42.344385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3b611-7f97-4762-9f7a-01bfffafcc7d None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:42.350140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 505/1031] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:42.355881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3b611-7f97-4762-9f7a-01bfffafcc7d None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:42.446484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7218f-c9fa-43d5-9ec7-5337c1d01456 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:42.459897 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3b611-7f97-4762-9f7a-01bfffafcc7d None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:42.484089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08f292-7545-461d-8cbe-57e95679e69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:42.484716 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08f292-7545-461d-8cbe-57e95679e69f None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:42.484977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08f292-7545-461d-8cbe-57e95679e69f None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:42.485247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08f292-7545-461d-8cbe-57e95679e69f None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:42.485907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08f292-7545-461d-8cbe-57e95679e69f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:42.486223 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08f292-7545-461d-8cbe-57e95679e69f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:42.487650 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08f292-7545-461d-8cbe-57e95679e69f None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:42.496129 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 527/1032] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Aug 23 07:00:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:42.496768 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ca306-4b51-4575-95e3-dc5e365e1d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:42.497377 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ca306-4b51-4575-95e3-dc5e365e1d0c None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:42.497647 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ca306-4b51-4575-95e3-dc5e365e1d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:42.497913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ca306-4b51-4575-95e3-dc5e365e1d0c None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:42.498539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ca306-4b51-4575-95e3-dc5e365e1d0c None admin] RBAC: Adding query filter params (type=image) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:42.498835 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ca306-4b51-4575-95e3-dc5e365e1d0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:42.500243 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ca306-4b51-4575-95e3-dc5e365e1d0c None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:42.501498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08f292-7545-461d-8cbe-57e95679e69f None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:42.505692 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 506/1033] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:42.511529 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ca306-4b51-4575-95e3-dc5e365e1d0c None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:42.518089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:42.545278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:42.605935 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08f292-7545-461d-8cbe-57e95679e69f None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:42.621792 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ca306-4b51-4575-95e3-dc5e365e1d0c None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:42.642714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798cc5a8-677f-40bb-8902-d502fdde03be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:42.643344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cc5a8-677f-40bb-8902-d502fdde03be None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:42.643709 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cc5a8-677f-40bb-8902-d502fdde03be None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:42.643996 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cc5a8-677f-40bb-8902-d502fdde03be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:42.658449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798cc5a8-677f-40bb-8902-d502fdde03be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:42.659956 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798cc5a8-677f-40bb-8902-d502fdde03be None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:42.660061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90e189-fb3c-410b-9c23-45ba4163411e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:42.660626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90e189-fb3c-410b-9c23-45ba4163411e None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:42.660907 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90e189-fb3c-410b-9c23-45ba4163411e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:42.661178 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90e189-fb3c-410b-9c23-45ba4163411e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:42.661893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90e189-fb3c-410b-9c23-45ba4163411e None admin] RBAC: Adding query filter params (interface=public, service_id=da7c4fbaa4144d8b9787d241058f1498, region_id=RegionOne) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:42.662202 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90e189-fb3c-410b-9c23-45ba4163411e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:42.663662 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90e189-fb3c-410b-9c23-45ba4163411e None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:42.673730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 507/1034] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Aug 23 07:00:42 2022] GET /identity/v3/endpoints?service_id=da7c4fbaa4144d8b9787d241058f1498&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:42.674381 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90e189-fb3c-410b-9c23-45ba4163411e None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:42.675154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-798cc5a8-677f-40bb-8902-d502fdde03be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:42.675154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:42.675154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:42.675154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:42.675154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:42.675154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:42.675154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:42.675154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:42.675154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:42.675154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:42.675154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:42.675154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:42.675154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:42.675154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:42.675154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:42.675154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:42.676099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:42.676099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:42.676099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:42.676099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:42.676099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:42.676099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:42.676099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:42.676099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:42.676099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:42.676099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:42.676099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:42.676099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:42.676099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:42.676099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:42.676099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:42.676099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:42.676099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:42.677007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:42.677007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:42.677007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:42.677007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 528/1035] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:42.682183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798cc5a8-677f-40bb-8902-d502fdde03be None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:42.749567 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:42.778899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:42.785519 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90e189-fb3c-410b-9c23-45ba4163411e None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:42.788509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798cc5a8-677f-40bb-8902-d502fdde03be None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:42.820746 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5fc9d-7d01-4030-9ce2-4fc21cff321e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:42.821357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5fc9d-7d01-4030-9ce2-4fc21cff321e None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:42.821636 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5fc9d-7d01-4030-9ce2-4fc21cff321e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:42.821918 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5fc9d-7d01-4030-9ce2-4fc21cff321e None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:42.822531 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5fc9d-7d01-4030-9ce2-4fc21cff321e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:42.823510 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87816179-149d-443b-add0-3d138cfa9011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:42.823955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5fc9d-7d01-4030-9ce2-4fc21cff321e None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:42.824121 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87816179-149d-443b-add0-3d138cfa9011 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:42.824380 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87816179-149d-443b-add0-3d138cfa9011 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:42.824651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87816179-149d-443b-add0-3d138cfa9011 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:42.825294 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87816179-149d-443b-add0-3d138cfa9011 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:42.826510 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87816179-149d-443b-add0-3d138cfa9011 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:42.827382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87816179-149d-443b-add0-3d138cfa9011 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:42.831926 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 508/1036] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Aug 23 07:00:42 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:00:42.841243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 529/1037] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:42.847108 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5fc9d-7d01-4030-9ce2-4fc21cff321e None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:42.965232 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5fc9d-7d01-4030-9ce2-4fc21cff321e None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:43.009120 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbf99d-a612-47fb-8e44-898e9cb9920b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:43.009683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbf99d-a612-47fb-8e44-898e9cb9920b None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:43.009961 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbf99d-a612-47fb-8e44-898e9cb9920b None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:43.010234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbf99d-a612-47fb-8e44-898e9cb9920b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:43.018413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbf99d-a612-47fb-8e44-898e9cb9920b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:43.019910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbf99d-a612-47fb-8e44-898e9cb9920b None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:43.031990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-c4cbf99d-a612-47fb-8e44-898e9cb9920b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:00:43.031990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:43.031990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:43.031990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:43.031990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:43.031990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:43.031990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:43.031990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:43.031990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:43.031990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:43.031990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:43.031990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:43.031990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:00:43.031990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:00:43.031990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:00:43.031990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:00:43.033006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:43.033006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:43.033006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:00:43.033006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:43.033006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:00:43.033006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:43.033006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:43.033006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:43.033006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:43.033006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:43.033006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:43.033006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:43.033006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:43.033006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:43.033006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:43.033006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:43.033006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:43.033006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:43.033967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:43.033967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:43.033967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:00:43.033967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:00:43.033967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:00:43.033967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:00:43.033967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:00:43.033967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:00:43.033967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:00:43.033967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:43.033967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 509/1038] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Aug 23 07:00:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:43.039592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87816179-149d-443b-add0-3d138cfa9011 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:43.157346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87816179-149d-443b-add0-3d138cfa9011 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:43.186919 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:43.197454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da6a6f-baad-4ae7-98ba-9e2ff215e789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:43.198059 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da6a6f-baad-4ae7-98ba-9e2ff215e789 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:43.198331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da6a6f-baad-4ae7-98ba-9e2ff215e789 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:43.198595 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da6a6f-baad-4ae7-98ba-9e2ff215e789 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:43.199317 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da6a6f-baad-4ae7-98ba-9e2ff215e789 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:43.199651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da6a6f-baad-4ae7-98ba-9e2ff215e789 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:43.201076 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da6a6f-baad-4ae7-98ba-9e2ff215e789 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:43.219296 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 530/1039] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Aug 23 07:00:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:43.224951 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbf99d-a612-47fb-8e44-898e9cb9920b None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:43.250035 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:43.334749 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbf99d-a612-47fb-8e44-898e9cb9920b None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:43.369435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38cd98-66ef-42ab-8b04-b2d9ebfdd1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:43.370041 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38cd98-66ef-42ab-8b04-b2d9ebfdd1e1 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:43.370316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38cd98-66ef-42ab-8b04-b2d9ebfdd1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:43.370585 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38cd98-66ef-42ab-8b04-b2d9ebfdd1e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:43.438704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38cd98-66ef-42ab-8b04-b2d9ebfdd1e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:43.440197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38cd98-66ef-42ab-8b04-b2d9ebfdd1e1 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:43.463092 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:43.548990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 510/1040] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:00:43 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:00:43.554688 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da6a6f-baad-4ae7-98ba-9e2ff215e789 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:43.658521 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da6a6f-baad-4ae7-98ba-9e2ff215e789 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:43.695505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242e534-70c5-4d07-a757-2b84d153389a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:43.696118 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242e534-70c5-4d07-a757-2b84d153389a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:43.696393 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242e534-70c5-4d07-a757-2b84d153389a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:43.696657 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242e534-70c5-4d07-a757-2b84d153389a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:43.707577 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242e534-70c5-4d07-a757-2b84d153389a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:43.709013 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242e534-70c5-4d07-a757-2b84d153389a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:43.723635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-6242e534-70c5-4d07-a757-2b84d153389a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:43.723635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:43.723635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:43.723635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:43.723635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:43.723635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:43.723635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:43.723635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:43.723635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:43.723635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:43.723635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:43.723635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:43.723635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:43.723635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:43.723635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:43.723635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:43.724612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:43.724612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:43.724612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:43.724612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:43.724612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:43.724612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:43.724612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:43.724612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:43.724612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:43.724612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:43.724612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:43.724612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:43.724612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:43.724612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:43.724612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:43.724612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:43.724612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:43.725585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:43.725585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:43.725585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:43.727498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 531/1041] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:43.730521 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38cd98-66ef-42ab-8b04-b2d9ebfdd1e1 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:43.834566 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38cd98-66ef-42ab-8b04-b2d9ebfdd1e1 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:43.855956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:43.872116 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c31924-a048-4cf9-9973-d99c1b77af8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:43.872735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c31924-a048-4cf9-9973-d99c1b77af8a None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:43.873018 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c31924-a048-4cf9-9973-d99c1b77af8a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:43.873290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c31924-a048-4cf9-9973-d99c1b77af8a None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:43.873943 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c31924-a048-4cf9-9973-d99c1b77af8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:43.874248 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c31924-a048-4cf9-9973-d99c1b77af8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:43.875694 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c31924-a048-4cf9-9973-d99c1b77af8a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:43.887699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 511/1042] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:43.893220 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242e534-70c5-4d07-a757-2b84d153389a None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:43.930718 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:43.948098 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68be613-81aa-4f17-b057-7577888267d1 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:43.948499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68be613-81aa-4f17-b057-7577888267d1 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:43.948849 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68be613-81aa-4f17-b057-7577888267d1 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:43.950403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 512/1043] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:00:43.956715 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c2c46-ce9c-4434-a71c-7e7374b7f73e None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:43.957077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c2c46-ce9c-4434-a71c-7e7374b7f73e None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:43.957362 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c2c46-ce9c-4434-a71c-7e7374b7f73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:43.975250 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-224c2c46-ce9c-4434-a71c-7e7374b7f73e None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:43.976628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-224c2c46-ce9c-4434-a71c-7e7374b7f73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:44.003093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242e534-70c5-4d07-a757-2b84d153389a None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:44.040395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab612703-3e7b-4e7d-8a23-49dafe052300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:44.041030 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab612703-3e7b-4e7d-8a23-49dafe052300 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:44.041307 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab612703-3e7b-4e7d-8a23-49dafe052300 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:44.041579 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab612703-3e7b-4e7d-8a23-49dafe052300 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:44.048382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab612703-3e7b-4e7d-8a23-49dafe052300 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:44.049838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab612703-3e7b-4e7d-8a23-49dafe052300 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:44.054661 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 513/1044] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:44.060977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-ab612703-3e7b-4e7d-8a23-49dafe052300 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:44.060977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:44.060977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:44.060977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:44.060977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:44.060977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:44.060977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:44.060977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:44.060977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:44.060977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:44.060977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:44.060977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:44.060977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:00:44.060977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:00:44.060977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:00:44.061859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:00:44.061859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:44.061859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:44.061859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:44.061859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:44.061859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:44.061859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:44.061859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:44.061859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:44.061859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:44.061859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:44.061859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:44.061859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:44.061859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:44.061859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:44.061859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:44.061859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:44.062765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:00:44.062765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:00:44.062765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:00:44.062765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:00:44.062765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:00:44.062765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:00:44.062765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:44.062765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:44.062765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 532/1045] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:44.070103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c2c46-ce9c-4434-a71c-7e7374b7f73e None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:44.164768 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c2c46-ce9c-4434-a71c-7e7374b7f73e None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:44.176314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:44.201051 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312521fb-2472-42cc-8fc2-3395e3f07ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:44.201590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312521fb-2472-42cc-8fc2-3395e3f07ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:44.201870 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312521fb-2472-42cc-8fc2-3395e3f07ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:44.202146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312521fb-2472-42cc-8fc2-3395e3f07ab5 None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:44.202849 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312521fb-2472-42cc-8fc2-3395e3f07ab5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:44.203157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312521fb-2472-42cc-8fc2-3395e3f07ab5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:44.204642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312521fb-2472-42cc-8fc2-3395e3f07ab5 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:44.216822 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 514/1046] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:00:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:44.222819 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab612703-3e7b-4e7d-8a23-49dafe052300 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:44.318618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:44.325829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab612703-3e7b-4e7d-8a23-49dafe052300 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:44.328499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312521fb-2472-42cc-8fc2-3395e3f07ab5 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:44.366513 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2419c-661d-47a1-a067-706baeb5a4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:44.367292 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2419c-661d-47a1-a067-706baeb5a4cc None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:44.367605 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2419c-661d-47a1-a067-706baeb5a4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:44.367880 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2419c-661d-47a1-a067-706baeb5a4cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:44.368614 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2419c-661d-47a1-a067-706baeb5a4cc None admin] RBAC: Adding query filter params (role.id=6d1af19135484d85af7accf2fdeb3cab, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=a8d2cd93042c4583b63afe629d47fce0) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:44.368921 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2419c-661d-47a1-a067-706baeb5a4cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:44.370365 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2419c-661d-47a1-a067-706baeb5a4cc None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:44.383445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 533/1047] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:00:44 2022] GET /identity/v3/role_assignments?user.id=a8d2cd93042c4583b63afe629d47fce0&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=6d1af19135484d85af7accf2fdeb3cab => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:44.444537 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312521fb-2472-42cc-8fc2-3395e3f07ab5 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:44.482505 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c73a7-6c3a-406b-8ca6-ce4fb179aa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:44.483061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c73a7-6c3a-406b-8ca6-ce4fb179aa54 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:44.483333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c73a7-6c3a-406b-8ca6-ce4fb179aa54 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:44.483622 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c73a7-6c3a-406b-8ca6-ce4fb179aa54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:44.495451 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c73a7-6c3a-406b-8ca6-ce4fb179aa54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:00:44.500709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-184c73a7-6c3a-406b-8ca6-ce4fb179aa54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:00:44.500709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:44.500709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:44.500709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:44.500709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:44.500709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:44.500709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:44.500709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:44.500709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:44.500709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:44.500709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:44.500709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:44.500709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:00:44.500709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:00:44.500709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:00:44.500709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:00:44.501684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:00:44.501684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:00:44.501684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:00:44.501684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:00:44.501684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:44.501684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:44.501684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:44.501684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:44.501684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:44.501684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:44.501684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:44.501684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:44.501684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:44.501684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:44.501684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:44.501684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:44.502543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:44.502543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:44.502543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:44.502543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:44.502543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:00:44.502543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:00:44.502543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:00:44.502543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:00:44.502543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:00:44.502543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:00:44.502543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:00:44.502543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:44.508656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 515/1048] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:00:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:44.508823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2419c-661d-47a1-a067-706baeb5a4cc None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:44.617365 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2419c-661d-47a1-a067-706baeb5a4cc None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:44.657361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c9453-ff5d-49e5-b7f4-3894852e36d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:44.657988 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c9453-ff5d-49e5-b7f4-3894852e36d3 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:44.658265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c9453-ff5d-49e5-b7f4-3894852e36d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:44.658535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c9453-ff5d-49e5-b7f4-3894852e36d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:44.659205 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c9453-ff5d-49e5-b7f4-3894852e36d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:44.659538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c9453-ff5d-49e5-b7f4-3894852e36d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:44.660920 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c9453-ff5d-49e5-b7f4-3894852e36d3 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:44.670103 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 534/1049] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 07:00:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:44.676054 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c73a7-6c3a-406b-8ca6-ce4fb179aa54 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:44.709177 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:44.782292 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c73a7-6c3a-406b-8ca6-ce4fb179aa54 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:44.797315 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b3820-2b86-494c-9511-83398fa6f237 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:44.797811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b3820-2b86-494c-9511-83398fa6f237 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:44.798163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b3820-2b86-494c-9511-83398fa6f237 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:44.799639 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 535/1050] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:44.806177 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d97b3-b969-47be-95d3-2baba95c5708 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:44.806461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d97b3-b969-47be-95d3-2baba95c5708 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:44.806752 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d97b3-b969-47be-95d3-2baba95c5708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:44.817385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13fc2f-2337-4764-91c9-d60280244b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:44.818025 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13fc2f-2337-4764-91c9-d60280244b37 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:44.818304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13fc2f-2337-4764-91c9-d60280244b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:44.818578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13fc2f-2337-4764-91c9-d60280244b37 None admin] PATH_INFO: `/v3/users/d91fee721beb47bca999f0b5a1deae00` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:44.827506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-a39d97b3-b969-47be-95d3-2baba95c5708 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:44.828709 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a39d97b3-b969-47be-95d3-2baba95c5708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:44.834796 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13fc2f-2337-4764-91c9-d60280244b37 None admin] RBAC: Authorizing `identity:get_user(user_id=d91fee721beb47bca999f0b5a1deae00)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:44.836594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13fc2f-2337-4764-91c9-d60280244b37 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:44.838481 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 516/1051] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 07:00:44 2022] GET /identity/v3/users/d91fee721beb47bca999f0b5a1deae00 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:44.844219 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13fc2f-2337-4764-91c9-d60280244b37 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:44.848503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:44.904315 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 536/1052] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:44.935708 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:44.949562 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13fc2f-2337-4764-91c9-d60280244b37 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:44.984673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c90f77-deaa-4a12-8b33-a7493cb475b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:44.985300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c90f77-deaa-4a12-8b33-a7493cb475b1 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:44.985582 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c90f77-deaa-4a12-8b33-a7493cb475b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:44.985852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c90f77-deaa-4a12-8b33-a7493cb475b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:45.053324 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c90f77-deaa-4a12-8b33-a7493cb475b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:45.054780 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c90f77-deaa-4a12-8b33-a7493cb475b1 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:45.140721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:45.155076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 517/1053] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:00:44 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 07:00:45.160879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d97b3-b969-47be-95d3-2baba95c5708 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:45.180634 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c90f77-deaa-4a12-8b33-a7493cb475b1 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:45.266519 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d97b3-b969-47be-95d3-2baba95c5708 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:45.290327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c90f77-deaa-4a12-8b33-a7493cb475b1 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:45.302707 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593600e-5eec-4184-8ea2-52d85abc3c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:45.303445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593600e-5eec-4184-8ea2-52d85abc3c0d None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:45.303666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593600e-5eec-4184-8ea2-52d85abc3c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:45.303925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593600e-5eec-4184-8ea2-52d85abc3c0d None admin] PATH_INFO: `/v3/projects/2fdde1d5ddbd4deda7b56d272054e90f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:45.305172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1593600e-5eec-4184-8ea2-52d85abc3c0d None admin] RBAC: Authorizing `identity:get_project(project_id=2fdde1d5ddbd4deda7b56d272054e90f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:45.306620 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1593600e-5eec-4184-8ea2-52d85abc3c0d None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:45.308494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 537/1054] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 07:00:45 2022] GET /identity/v3/projects/2fdde1d5ddbd4deda7b56d272054e90f => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:45.314287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593600e-5eec-4184-8ea2-52d85abc3c0d None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:45.333169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461f224-290d-4df6-9ab3-ad9022984724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:45.333965 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461f224-290d-4df6-9ab3-ad9022984724 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:45.334248 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461f224-290d-4df6-9ab3-ad9022984724 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:45.334525 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461f224-290d-4df6-9ab3-ad9022984724 None admin] PATH_INFO: `/v3/services/image` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:45.335119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461f224-290d-4df6-9ab3-ad9022984724 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:45.337090 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461f224-290d-4df6-9ab3-ad9022984724 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:45.337619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:45.346293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-a461f224-290d-4df6-9ab3-ad9022984724 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:00:45.346293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:45.346293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:00:45.346293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:00:45.346293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:00:45.346293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:00:45.346293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:00:45.346293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:45.346293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:00:45.346293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:45.346293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:00:45.346293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:45.346293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:45.346293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:45.346293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:45.346293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:45.346293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:45.347316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:45.347316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:45.347316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:45.347316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:45.347316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:45.347316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:45.347316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:00:45.347316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:00:45.347316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:00:45.347316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:00:45.347316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:45.347316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:45.347316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:45.347316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:45.347316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:45.347316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:45.348196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:45.348196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:45.348196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:45.348196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:45.348196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:45.348196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:45.348196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:45.348196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:45.348196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:45.348196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:45.348196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:00:45.348196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:00:45.348196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:00:45.348196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:45.348196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 518/1055] 10.222.0.17 () {56 vars in 1105 bytes} [Tue Aug 23 07:00:45 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:45.353341 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461f224-290d-4df6-9ab3-ad9022984724 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:45.410262 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:45.429106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593600e-5eec-4184-8ea2-52d85abc3c0d None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:45.466547 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461f224-290d-4df6-9ab3-ad9022984724 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:45.467879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82ce9a-9378-4565-9410-94ac4a9513dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:45.468491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82ce9a-9378-4565-9410-94ac4a9513dc None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:45.468768 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82ce9a-9378-4565-9410-94ac4a9513dc None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:45.469060 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82ce9a-9378-4565-9410-94ac4a9513dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:45.469773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82ce9a-9378-4565-9410-94ac4a9513dc None admin] RBAC: Adding query filter params (role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.project.id=2fdde1d5ddbd4deda7b56d272054e90f, user.id=d91fee721beb47bca999f0b5a1deae00) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:45.470074 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82ce9a-9378-4565-9410-94ac4a9513dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:45.471536 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82ce9a-9378-4565-9410-94ac4a9513dc None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:45.477697 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 538/1056] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:00:45 2022] GET /identity/v3/role_assignments?user.id=d91fee721beb47bca999f0b5a1deae00&scope.project.id=2fdde1d5ddbd4deda7b56d272054e90f&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:45.504239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de3f0c-5dc9-49a1-968b-85ebdfde917e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:45.505186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de3f0c-5dc9-49a1-968b-85ebdfde917e None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:45.505665 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de3f0c-5dc9-49a1-968b-85ebdfde917e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:45.506102 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de3f0c-5dc9-49a1-968b-85ebdfde917e None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:45.506918 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de3f0c-5dc9-49a1-968b-85ebdfde917e None admin] RBAC: Adding query filter params (name=image) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:45.507489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de3f0c-5dc9-49a1-968b-85ebdfde917e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:45.509188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de3f0c-5dc9-49a1-968b-85ebdfde917e None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:45.515713 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 519/1057] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:45.521373 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82ce9a-9378-4565-9410-94ac4a9513dc None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:45.618094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:45.636370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82ce9a-9378-4565-9410-94ac4a9513dc None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:45.670819 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945fc12-72e5-465d-a7bd-6dc77136aa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:45.671453 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945fc12-72e5-465d-a7bd-6dc77136aa6c None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:45.671742 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945fc12-72e5-465d-a7bd-6dc77136aa6c None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:45.672014 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945fc12-72e5-465d-a7bd-6dc77136aa6c None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:45.672657 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945fc12-72e5-465d-a7bd-6dc77136aa6c None admin] RBAC: Adding query filter params (type=image) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:45.672961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945fc12-72e5-465d-a7bd-6dc77136aa6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:45.674339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945fc12-72e5-465d-a7bd-6dc77136aa6c None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:45.684174 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 539/1058] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:45.685511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de3f0c-5dc9-49a1-968b-85ebdfde917e None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:45.791812 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de3f0c-5dc9-49a1-968b-85ebdfde917e None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:45.809239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:45.827731 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce29dc7-1158-450b-be31-a2cbae022f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:45.828349 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce29dc7-1158-450b-be31-a2cbae022f37 None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:45.828626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce29dc7-1158-450b-be31-a2cbae022f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:45.828896 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce29dc7-1158-450b-be31-a2cbae022f37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:45.829576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce29dc7-1158-450b-be31-a2cbae022f37 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:45.830950 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce29dc7-1158-450b-be31-a2cbae022f37 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:45.859446 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 520/1059] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:00:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:00:46.308128 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d17742-4477-44e2-a9e6-3ab580ed3512 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:46.308508 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d17742-4477-44e2-a9e6-3ab580ed3512 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:46.308673 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d17742-4477-44e2-a9e6-3ab580ed3512 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:46.310303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 540/1060] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:46.315489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1425c62-5617-40f2-97d7-c6ffceca7182 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:46.315780 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1425c62-5617-40f2-97d7-c6ffceca7182 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:46.316050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1425c62-5617-40f2-97d7-c6ffceca7182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:46.335352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-e1425c62-5617-40f2-97d7-c6ffceca7182 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:46.336576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e1425c62-5617-40f2-97d7-c6ffceca7182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:46.415578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 521/1061] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:46.682469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d17742-4477-44e2-a9e6-3ab580ed3512 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:46.785876 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d17742-4477-44e2-a9e6-3ab580ed3512 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:46.820777 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740c979-e371-4cd7-bec7-4c7def2e3a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:46.821461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740c979-e371-4cd7-bec7-4c7def2e3a8f None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:46.821739 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740c979-e371-4cd7-bec7-4c7def2e3a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:46.822012 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740c979-e371-4cd7-bec7-4c7def2e3a8f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:46.833461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740c979-e371-4cd7-bec7-4c7def2e3a8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:00:46.838358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-0740c979-e371-4cd7-bec7-4c7def2e3a8f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:00:46.838358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:46.838358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:46.838358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:46.838358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:46.838358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:46.838358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:46.838358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:46.838358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:46.838358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:46.838358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:46.838358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:46.838358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:00:46.838358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:00:46.838358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:00:46.838358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:00:46.839282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:00:46.839282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:00:46.839282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:00:46.839282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:00:46.839282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:46.839282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:46.839282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:46.839282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:46.839282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:46.839282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:46.839282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:46.839282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:46.839282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:46.839282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:46.839282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:46.839282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:46.840176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:46.840176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:46.840176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:46.840176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:46.840176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:00:46.840176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:00:46.840176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:00:46.840176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:00:46.840176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:00:46.840176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:00:46.840176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:00:46.840176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:46.843737 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 541/1062] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Aug 23 07:00:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:46.845413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1425c62-5617-40f2-97d7-c6ffceca7182 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:46.945489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:46.950085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1425c62-5617-40f2-97d7-c6ffceca7182 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:46.990096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdedb26-fd5e-4582-8226-4766d20b27f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:46.990706 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdedb26-fd5e-4582-8226-4766d20b27f0 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:46.990983 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdedb26-fd5e-4582-8226-4766d20b27f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:46.991250 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdedb26-fd5e-4582-8226-4766d20b27f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:46.992036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdedb26-fd5e-4582-8226-4766d20b27f0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:46.992477 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdedb26-fd5e-4582-8226-4766d20b27f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:46.993896 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdedb26-fd5e-4582-8226-4766d20b27f0 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:47.002340 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 522/1063] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Aug 23 07:00:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:47.008168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740c979-e371-4cd7-bec7-4c7def2e3a8f None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:47.020879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:47.114718 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740c979-e371-4cd7-bec7-4c7def2e3a8f None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:47.149166 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6a894-046f-4fa6-8f1d-e8f04cc9752a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:47.149799 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6a894-046f-4fa6-8f1d-e8f04cc9752a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:47.150071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6a894-046f-4fa6-8f1d-e8f04cc9752a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:47.150337 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6a894-046f-4fa6-8f1d-e8f04cc9752a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:47.161192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6a894-046f-4fa6-8f1d-e8f04cc9752a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:47.162644 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6a894-046f-4fa6-8f1d-e8f04cc9752a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:47.177098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-3fc6a894-046f-4fa6-8f1d-e8f04cc9752a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:47.177098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:47.177098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:47.177098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:47.177098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:47.177098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:47.177098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:47.177098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:47.177098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:47.177098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:47.177098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:47.177098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:47.177098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:47.177098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:47.177098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:47.177098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:47.178069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:47.178069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:47.178069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:47.178069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:47.178069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:47.178069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:47.178069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:47.178069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:47.178069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:47.178069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:47.178069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:47.178069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:47.178069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:47.178069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:47.178069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:47.178069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:47.178069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:47.178981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:47.178981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:47.178981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:47.179815 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 542/1064] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:47.184224 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdedb26-fd5e-4582-8226-4766d20b27f0 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:47.294957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdedb26-fd5e-4582-8226-4766d20b27f0 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:47.330467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42c391-5bec-487e-9589-4c55db2ff39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:47.331157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42c391-5bec-487e-9589-4c55db2ff39f None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:47.331435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42c391-5bec-487e-9589-4c55db2ff39f None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:47.332031 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42c391-5bec-487e-9589-4c55db2ff39f None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:47.332694 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42c391-5bec-487e-9589-4c55db2ff39f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:47.333058 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42c391-5bec-487e-9589-4c55db2ff39f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:47.334443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42c391-5bec-487e-9589-4c55db2ff39f None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:47.351710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 523/1065] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:47.353100 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6a894-046f-4fa6-8f1d-e8f04cc9752a None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:47.368204 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38d026-84ca-4f08-8088-8fa6eb5cbb46 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:47.368492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38d026-84ca-4f08-8088-8fa6eb5cbb46 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:47.368774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38d026-84ca-4f08-8088-8fa6eb5cbb46 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:47.370259 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 524/1066] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:00:47.374964 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc50de-6a02-44f8-a269-49d8ce6b5f98 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:47.375246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc50de-6a02-44f8-a269-49d8ce6b5f98 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:47.375551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc50de-6a02-44f8-a269-49d8ce6b5f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:47.393778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-9bdc50de-6a02-44f8-a269-49d8ce6b5f98 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:47.394964 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9bdc50de-6a02-44f8-a269-49d8ce6b5f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:47.410592 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:47.422157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:47.467803 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6a894-046f-4fa6-8f1d-e8f04cc9752a None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:47.471331 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 525/1067] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:00:47.502663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cab72-e333-4a34-b136-3023e515764e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:47.503293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cab72-e333-4a34-b136-3023e515764e None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:47.503612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cab72-e333-4a34-b136-3023e515764e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:47.503883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cab72-e333-4a34-b136-3023e515764e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:47.511638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24cab72-e333-4a34-b136-3023e515764e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:47.513073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24cab72-e333-4a34-b136-3023e515764e None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:47.524171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-e24cab72-e333-4a34-b136-3023e515764e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:00:47.524171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:47.524171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:47.524171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:47.524171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:47.524171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:47.524171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:47.524171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:47.524171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:47.524171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:47.524171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:47.524171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:47.524171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:00:47.524171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:00:47.524171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:00:47.524171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:00:47.525163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:47.525163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:47.525163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:00:47.525163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:47.525163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:00:47.525163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:47.525163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:47.525163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:47.525163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:47.525163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:47.525163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:47.525163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:47.525163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:47.525163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:47.525163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:47.525163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:47.525163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:47.525163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:47.526131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:47.526131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:47.526131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:00:47.526131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:00:47.526131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:00:47.526131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:00:47.526131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:00:47.526131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:00:47.526131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:00:47.526131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:47.526131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 543/1068] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Aug 23 07:00:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:47.531061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc50de-6a02-44f8-a269-49d8ce6b5f98 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:47.633309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc50de-6a02-44f8-a269-49d8ce6b5f98 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:47.669238 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12203240-a039-42da-9954-f4f57589bf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:47.669864 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12203240-a039-42da-9954-f4f57589bf7c None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:47.670144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12203240-a039-42da-9954-f4f57589bf7c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:47.670481 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12203240-a039-42da-9954-f4f57589bf7c None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:47.671222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12203240-a039-42da-9954-f4f57589bf7c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:47.671552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12203240-a039-42da-9954-f4f57589bf7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:47.673003 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12203240-a039-42da-9954-f4f57589bf7c None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:47.694722 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 526/1069] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Aug 23 07:00:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:47.700560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cab72-e333-4a34-b136-3023e515764e None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:47.738410 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12203240-a039-42da-9954-f4f57589bf7c None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:47.752530 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:47.812913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cab72-e333-4a34-b136-3023e515764e None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:47.833118 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:47.845146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12203240-a039-42da-9954-f4f57589bf7c None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:47.848848 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648de17e-4c88-4004-884c-2dba356be775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:47.849482 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648de17e-4c88-4004-884c-2dba356be775 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:47.849764 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648de17e-4c88-4004-884c-2dba356be775 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:47.850038 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648de17e-4c88-4004-884c-2dba356be775 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:47.860723 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648de17e-4c88-4004-884c-2dba356be775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:47.862169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648de17e-4c88-4004-884c-2dba356be775 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:47.876228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-648de17e-4c88-4004-884c-2dba356be775 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:47.876228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:47.876228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:47.876228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:47.876228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:47.876228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:47.876228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:47.876228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:47.876228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:47.876228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:47.876228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:47.876228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:47.876228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:47.876228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:47.876228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:47.876228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:47.877361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:47.877361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:47.877361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:47.877361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:47.877361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:47.877361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:47.877361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:47.877361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:47.877361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:47.877361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:47.877361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:47.877361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:47.877361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:47.877361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:47.877361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:47.877361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:47.877361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:47.878284 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:47.878284 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:47.878284 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:47.878284 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 544/1070] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:47.878712 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec65e55-4c47-4513-ab9e-ca98a00f1dc8 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:47.878993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec65e55-4c47-4513-ab9e-ca98a00f1dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:47.879262 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec65e55-4c47-4513-ab9e-ca98a00f1dc8 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:47.880526 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 545/1071] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:47.883051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec65e55-4c47-4513-ab9e-ca98a00f1dc8 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:47.884371 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08d4cc-e682-45cf-a97f-f4b070a9bc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:47.884965 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08d4cc-e682-45cf-a97f-f4b070a9bc03 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:47.885242 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08d4cc-e682-45cf-a97f-f4b070a9bc03 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:47.885664 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08d4cc-e682-45cf-a97f-f4b070a9bc03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:47.896032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08d4cc-e682-45cf-a97f-f4b070a9bc03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:00:47.901112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-2b08d4cc-e682-45cf-a97f-f4b070a9bc03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:00:47.901112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:47.901112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:47.901112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:47.901112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:47.901112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:47.901112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:47.901112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:47.901112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:47.901112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:47.901112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:47.901112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:47.901112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:00:47.901112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:00:47.901112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:00:47.901112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:00:47.902077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:00:47.902077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:00:47.902077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:00:47.902077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:00:47.902077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:47.902077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:47.902077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:47.902077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:47.902077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:47.902077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:47.902077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:47.902077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:47.902077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:47.902077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:47.902077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:47.902077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:47.902920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:47.902920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:47.902920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:47.902920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:47.902920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:00:47.902920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:00:47.902920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:00:47.902920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:00:47.902920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:00:47.902920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:00:47.902920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:00:47.902920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:47.902920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 527/1072] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:00:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:47.903665 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca2dc8-660a-4636-897a-169eae663d92 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:47.904076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca2dc8-660a-4636-897a-169eae663d92 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:47.904076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca2dc8-660a-4636-897a-169eae663d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:47.922707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-a8ca2dc8-660a-4636-897a-169eae663d92 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:47.923924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-a8ca2dc8-660a-4636-897a-169eae663d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:47.985060 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:47.988926 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec65e55-4c47-4513-ab9e-ca98a00f1dc8 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:47.991074 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 528/1073] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:47.992125 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca2dc8-660a-4636-897a-169eae663d92 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:48.034702 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9293a-2490-4b67-aa9b-2e11c54918ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:48.035318 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9293a-2490-4b67-aa9b-2e11c54918ae None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:48.035718 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9293a-2490-4b67-aa9b-2e11c54918ae None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:48.035994 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9293a-2490-4b67-aa9b-2e11c54918ae None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:48.036757 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9293a-2490-4b67-aa9b-2e11c54918ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:48.037060 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9293a-2490-4b67-aa9b-2e11c54918ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:48.038459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9293a-2490-4b67-aa9b-2e11c54918ae None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:48.051842 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 546/1074] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:48.057366 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9293a-2490-4b67-aa9b-2e11c54918ae None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:48.088971 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:48.089412 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca2dc8-660a-4636-897a-169eae663d92 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:48.124392 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d2b99-e952-4001-a30a-b66672374dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:48.125002 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d2b99-e952-4001-a30a-b66672374dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:48.125276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d2b99-e952-4001-a30a-b66672374dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:48.125546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d2b99-e952-4001-a30a-b66672374dc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:48.126202 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d2b99-e952-4001-a30a-b66672374dc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:48.126499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d2b99-e952-4001-a30a-b66672374dc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:48.127919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d2b99-e952-4001-a30a-b66672374dc9 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:48.136416 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 529/1075] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 07:00:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:48.142110 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d2b99-e952-4001-a30a-b66672374dc9 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:48.165320 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9293a-2490-4b67-aa9b-2e11c54918ae None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:48.201565 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d859ca0-b36d-45dd-a1f0-67e693006e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:48.202290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d859ca0-b36d-45dd-a1f0-67e693006e4b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:48.202563 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d859ca0-b36d-45dd-a1f0-67e693006e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:48.202866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d859ca0-b36d-45dd-a1f0-67e693006e4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:48.206479 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:48.210287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d859ca0-b36d-45dd-a1f0-67e693006e4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:48.211903 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d859ca0-b36d-45dd-a1f0-67e693006e4b None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:48.223649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-2d859ca0-b36d-45dd-a1f0-67e693006e4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:48.223649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:48.223649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:48.223649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:48.223649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:48.223649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:48.223649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:48.223649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:48.223649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:48.223649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:48.223649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:48.223649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:48.223649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:00:48.223649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:00:48.223649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:00:48.224561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:00:48.224561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:48.224561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:48.224561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:48.224561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:48.224561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:48.224561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:48.224561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:48.224561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:48.224561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:48.224561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:48.224561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:48.224561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:48.224561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:48.224561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:48.224561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:48.224561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:48.225532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:00:48.225532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:00:48.225532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:00:48.225532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:00:48.225532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:00:48.225532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:00:48.225532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:48.225532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:48.225532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 547/1076] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:48.230775 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d859ca0-b36d-45dd-a1f0-67e693006e4b None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:48.258841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d2b99-e952-4001-a30a-b66672374dc9 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:48.294216 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47acaf6f-830d-44a7-82c0-20f6cbf50b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:48.294846 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acaf6f-830d-44a7-82c0-20f6cbf50b65 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:48.295124 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acaf6f-830d-44a7-82c0-20f6cbf50b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:48.295393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acaf6f-830d-44a7-82c0-20f6cbf50b65 None admin] PATH_INFO: `/v3/users/d91fee721beb47bca999f0b5a1deae00` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:48.296712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47acaf6f-830d-44a7-82c0-20f6cbf50b65 None admin] RBAC: Authorizing `identity:get_user(user_id=d91fee721beb47bca999f0b5a1deae00)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:48.297988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47acaf6f-830d-44a7-82c0-20f6cbf50b65 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:48.299913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 530/1077] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 07:00:48 2022] GET /identity/v3/users/d91fee721beb47bca999f0b5a1deae00 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:48.300678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47acaf6f-830d-44a7-82c0-20f6cbf50b65 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:48.329838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:48.338712 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d859ca0-b36d-45dd-a1f0-67e693006e4b None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:48.374765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f39c0-bcd7-4d0e-8932-8105342c62c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:48.375372 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f39c0-bcd7-4d0e-8932-8105342c62c7 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:48.375679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f39c0-bcd7-4d0e-8932-8105342c62c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:48.375951 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f39c0-bcd7-4d0e-8932-8105342c62c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:48.376643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f39c0-bcd7-4d0e-8932-8105342c62c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:48.376942 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f39c0-bcd7-4d0e-8932-8105342c62c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:48.378380 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f39c0-bcd7-4d0e-8932-8105342c62c7 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:48.390328 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 548/1078] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:00:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:48.391041 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f39c0-bcd7-4d0e-8932-8105342c62c7 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:48.404546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47acaf6f-830d-44a7-82c0-20f6cbf50b65 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:48.431730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:48.440027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146eb041-bf78-4b90-b2dc-22b47d1a0d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:48.440631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146eb041-bf78-4b90-b2dc-22b47d1a0d05 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:48.440911 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146eb041-bf78-4b90-b2dc-22b47d1a0d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:48.441174 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146eb041-bf78-4b90-b2dc-22b47d1a0d05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:48.489200 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f39c0-bcd7-4d0e-8932-8105342c62c7 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:48.508193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146eb041-bf78-4b90-b2dc-22b47d1a0d05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:48.509667 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146eb041-bf78-4b90-b2dc-22b47d1a0d05 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:48.522697 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74af2d1-7a9c-4d73-ba95-f2c14e911e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:48.523492 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74af2d1-7a9c-4d73-ba95-f2c14e911e64 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:48.523868 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74af2d1-7a9c-4d73-ba95-f2c14e911e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:48.524235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74af2d1-7a9c-4d73-ba95-f2c14e911e64 None admin] PATH_INFO: `/v3/projects/2fdde1d5ddbd4deda7b56d272054e90f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:48.525735 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74af2d1-7a9c-4d73-ba95-f2c14e911e64 None admin] RBAC: Authorizing `identity:get_project(project_id=2fdde1d5ddbd4deda7b56d272054e90f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:48.527391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74af2d1-7a9c-4d73-ba95-f2c14e911e64 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:48.529032 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 549/1079] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 07:00:48 2022] GET /identity/v3/projects/2fdde1d5ddbd4deda7b56d272054e90f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:48.529814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74af2d1-7a9c-4d73-ba95-f2c14e911e64 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:48.556525 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:48.611009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 531/1080] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:00:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:00:48.611867 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146eb041-bf78-4b90-b2dc-22b47d1a0d05 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:48.638729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:48.641358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74af2d1-7a9c-4d73-ba95-f2c14e911e64 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:48.677380 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aeac28-e9e6-4d13-bb29-318100b43f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:48.678040 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeac28-e9e6-4d13-bb29-318100b43f77 None admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:48.678312 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeac28-e9e6-4d13-bb29-318100b43f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:48.678582 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeac28-e9e6-4d13-bb29-318100b43f77 None admin] PATH_INFO: `/v3/projects/351a6777525c4207826d7df0d3768e99/users/a8d2cd93042c4583b63afe629d47fce0/roles/6d1af19135484d85af7accf2fdeb3cab` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:48.681257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aeac28-e9e6-4d13-bb29-318100b43f77 None admin] RBAC: Authorizing `identity:create_grant(project_id=351a6777525c4207826d7df0d3768e99, user_id=a8d2cd93042c4583b63afe629d47fce0, role_id=6d1af19135484d85af7accf2fdeb3cab)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:48.683633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aeac28-e9e6-4d13-bb29-318100b43f77 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:48.697359 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 550/1081] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 07:00:48 2022] PUT /identity/v3/projects/351a6777525c4207826d7df0d3768e99/users/a8d2cd93042c4583b63afe629d47fce0/roles/6d1af19135484d85af7accf2fdeb3cab => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:00:48.698145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aeac28-e9e6-4d13-bb29-318100b43f77 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:48.725160 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146eb041-bf78-4b90-b2dc-22b47d1a0d05 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:48.766856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c6fb9-5802-4227-b973-c52522e0f12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:48.767955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c6fb9-5802-4227-b973-c52522e0f12a None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:48.768412 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c6fb9-5802-4227-b973-c52522e0f12a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:48.768861 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c6fb9-5802-4227-b973-c52522e0f12a None admin] PATH_INFO: `/v3/projects/2fdde1d5ddbd4deda7b56d272054e90f/users/d91fee721beb47bca999f0b5a1deae00/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:48.771359 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c6fb9-5802-4227-b973-c52522e0f12a None admin] RBAC: Authorizing `identity:create_grant(project_id=2fdde1d5ddbd4deda7b56d272054e90f, user_id=d91fee721beb47bca999f0b5a1deae00, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:48.773295 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c6fb9-5802-4227-b973-c52522e0f12a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:48.787027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:48.792533 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 532/1082] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 07:00:48 2022] PUT /identity/v3/projects/2fdde1d5ddbd4deda7b56d272054e90f/users/d91fee721beb47bca999f0b5a1deae00/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:00:48.823885 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aeac28-e9e6-4d13-bb29-318100b43f77 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:48.872714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011aaa3-a192-4814-8b46-3fcc75b608a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:48.873367 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011aaa3-a192-4814-8b46-3fcc75b608a2 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:48.873642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011aaa3-a192-4814-8b46-3fcc75b608a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:48.873907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011aaa3-a192-4814-8b46-3fcc75b608a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:48.885934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011aaa3-a192-4814-8b46-3fcc75b608a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:48.887397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011aaa3-a192-4814-8b46-3fcc75b608a2 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:48.905218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-1011aaa3-a192-4814-8b46-3fcc75b608a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:48.905218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:48.905218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:48.905218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:48.905218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:48.905218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:48.905218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:48.905218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:48.905218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:48.905218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:48.905218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:48.905218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:48.905218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:48.905218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:48.905218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:48.905218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:48.906219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:48.906219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:48.906219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:48.906219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:48.906219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:48.906219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:48.906219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:48.906219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:48.906219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:48.906219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:48.906219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:48.906219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:48.906219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:48.906219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:48.906219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:48.906219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:48.906219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:48.907134 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:48.907134 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:48.907134 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:48.907134 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 551/1083] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:48.912693 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c6fb9-5802-4227-b973-c52522e0f12a None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:49.010579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:49.025360 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c6fb9-5802-4227-b973-c52522e0f12a None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:49.063249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1560a-e7cc-4663-b78b-f62a15a062c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:49.063967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1560a-e7cc-4663-b78b-f62a15a062c2 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:49.064313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1560a-e7cc-4663-b78b-f62a15a062c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:49.064652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1560a-e7cc-4663-b78b-f62a15a062c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:49.065366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1560a-e7cc-4663-b78b-f62a15a062c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:49.065736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1560a-e7cc-4663-b78b-f62a15a062c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:49.067185 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1560a-e7cc-4663-b78b-f62a15a062c2 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:49.079443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 533/1084] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:50.592283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ecaf6-68b6-4622-bc0b-33bec5e464c7 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:50.592882 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ecaf6-68b6-4622-bc0b-33bec5e464c7 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:50.592882 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ecaf6-68b6-4622-bc0b-33bec5e464c7 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:50.594164 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 552/1085] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:50.599519 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991e1a6-0d4c-4012-81b3-bc1aef552410 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:50.599939 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991e1a6-0d4c-4012-81b3-bc1aef552410 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:50.600071 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991e1a6-0d4c-4012-81b3-bc1aef552410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:50.620063 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-b991e1a6-0d4c-4012-81b3-bc1aef552410 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:50.621259 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b991e1a6-0d4c-4012-81b3-bc1aef552410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:50.694241 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 534/1086] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:50.700885 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc51f2-6520-4d5d-9b10-17efaf4cb508 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:50.701170 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc51f2-6520-4d5d-9b10-17efaf4cb508 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:50.701449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc51f2-6520-4d5d-9b10-17efaf4cb508 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:50.702794 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 553/1087] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:50.708403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87156a85-7ce9-4979-8354-879f5294a852 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:50.708691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87156a85-7ce9-4979-8354-879f5294a852 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:50.708973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87156a85-7ce9-4979-8354-879f5294a852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:50.729088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-87156a85-7ce9-4979-8354-879f5294a852 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:50.730281 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-87156a85-7ce9-4979-8354-879f5294a852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:50.797777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 535/1088] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:50.958264 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc51f2-6520-4d5d-9b10-17efaf4cb508 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:51.063221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87156a85-7ce9-4979-8354-879f5294a852 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:51.063438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc51f2-6520-4d5d-9b10-17efaf4cb508 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:51.074981 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:51.095177 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:51.100777 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292705f-3cd8-4d06-b3b5-78a8a8763c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:51.101463 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292705f-3cd8-4d06-b3b5-78a8a8763c5a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:51.101740 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292705f-3cd8-4d06-b3b5-78a8a8763c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:51.102020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292705f-3cd8-4d06-b3b5-78a8a8763c5a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:51.112644 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292705f-3cd8-4d06-b3b5-78a8a8763c5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:00:51.117734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-e292705f-3cd8-4d06-b3b5-78a8a8763c5a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:00:51.117734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:51.117734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:51.117734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:51.117734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:51.117734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:51.117734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:51.117734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:51.117734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:51.117734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:51.117734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:51.117734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:51.117734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:00:51.117734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:00:51.117734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:00:51.117734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:00:51.118685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:00:51.118685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:00:51.118685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:00:51.118685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:00:51.118685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:51.118685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:51.118685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:51.118685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:51.118685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:51.118685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:51.118685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:51.118685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:51.118685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:51.118685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:51.118685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:51.118685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:51.119566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:51.119566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:51.119566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:51.119566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:51.119566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:00:51.119566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:00:51.119566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:00:51.119566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:00:51.119566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:00:51.119566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:00:51.119566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:00:51.119566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:51.119566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 554/1089] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Aug 23 07:00:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:51.124690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292705f-3cd8-4d06-b3b5-78a8a8763c5a None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:51.166277 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87156a85-7ce9-4979-8354-879f5294a852 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:51.198965 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152093d6-5ab8-45b4-858b-0d58c3d4877f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:51.199619 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152093d6-5ab8-45b4-858b-0d58c3d4877f None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:51.199892 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152093d6-5ab8-45b4-858b-0d58c3d4877f None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:51.200162 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152093d6-5ab8-45b4-858b-0d58c3d4877f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:51.210688 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152093d6-5ab8-45b4-858b-0d58c3d4877f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:00:51.215579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-152093d6-5ab8-45b4-858b-0d58c3d4877f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:00:51.215579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:51.215579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:51.215579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:51.215579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:51.215579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:51.215579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:51.215579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:51.215579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:51.215579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:51.215579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:51.215579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:51.215579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:00:51.215579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:00:51.215579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:00:51.215579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:00:51.216551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:00:51.216551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:00:51.216551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:00:51.216551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:00:51.216551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:51.216551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:51.216551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:51.216551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:51.216551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:51.216551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:51.216551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:51.216551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:51.216551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:51.216551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:51.216551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:51.216551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:51.217572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:51.217572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:51.217572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:51.217572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:51.217572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:00:51.217572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:00:51.217572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:00:51.217572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:00:51.217572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:00:51.217572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:00:51.217572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:00:51.217572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:51.217572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 536/1090] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:00:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:51.222598 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152093d6-5ab8-45b4-858b-0d58c3d4877f None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:51.222722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292705f-3cd8-4d06-b3b5-78a8a8763c5a None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:51.257479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e7728-337e-468e-a46d-3c0d914b4e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:51.258334 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e7728-337e-468e-a46d-3c0d914b4e13 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:51.258614 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e7728-337e-468e-a46d-3c0d914b4e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:51.258884 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e7728-337e-468e-a46d-3c0d914b4e13 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:51.259635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e7728-337e-468e-a46d-3c0d914b4e13 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:51.259935 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e7728-337e-468e-a46d-3c0d914b4e13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:51.261299 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e7728-337e-468e-a46d-3c0d914b4e13 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:51.269346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 555/1091] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Aug 23 07:00:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:51.275251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e7728-337e-468e-a46d-3c0d914b4e13 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:51.290268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:51.324613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152093d6-5ab8-45b4-858b-0d58c3d4877f None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:51.329709 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:51.358273 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d0607-2073-4c64-a732-8008ed4dc070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:51.358873 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d0607-2073-4c64-a732-8008ed4dc070 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:51.359147 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d0607-2073-4c64-a732-8008ed4dc070 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:51.359443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d0607-2073-4c64-a732-8008ed4dc070 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:51.360097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d0607-2073-4c64-a732-8008ed4dc070 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:51.360392 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d0607-2073-4c64-a732-8008ed4dc070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:51.361764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d0607-2073-4c64-a732-8008ed4dc070 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:51.370293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 537/1092] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 07:00:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:51.376607 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d0607-2073-4c64-a732-8008ed4dc070 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:51.379157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e7728-337e-468e-a46d-3c0d914b4e13 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:51.413389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b236e5-a445-4a9c-9892-ea22115bace0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:51.414172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b236e5-a445-4a9c-9892-ea22115bace0 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:51.414450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b236e5-a445-4a9c-9892-ea22115bace0 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:51.414718 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b236e5-a445-4a9c-9892-ea22115bace0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:51.425458 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b236e5-a445-4a9c-9892-ea22115bace0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:51.426910 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b236e5-a445-4a9c-9892-ea22115bace0 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:51.441026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-c9b236e5-a445-4a9c-9892-ea22115bace0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:51.441026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:51.441026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:51.441026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:51.441026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:51.441026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:51.441026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:51.441026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:51.441026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:51.441026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:51.441026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:51.441026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:51.441026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:51.441026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:51.441026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:51.441026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:51.442188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:51.442188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:51.442188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:51.442188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:51.442188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:51.442188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:51.442188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:51.442188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:51.442188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:51.442188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:51.442188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:51.442188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:51.442188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:51.442188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:51.442188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:51.442188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:51.442188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:51.443292 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:51.443292 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:51.443292 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:51.444863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 556/1093] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:51.450640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b236e5-a445-4a9c-9892-ea22115bace0 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:51.482874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d0607-2073-4c64-a732-8008ed4dc070 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:51.510995 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:51.517346 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccad71-4d12-4a3c-ad54-19b41f68ab67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:51.517975 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccad71-4d12-4a3c-ad54-19b41f68ab67 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:51.518249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccad71-4d12-4a3c-ad54-19b41f68ab67 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:51.518518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccad71-4d12-4a3c-ad54-19b41f68ab67 None admin] PATH_INFO: `/v3/users/d91fee721beb47bca999f0b5a1deae00` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:51.519695 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bccad71-4d12-4a3c-ad54-19b41f68ab67 None admin] RBAC: Authorizing `identity:get_user(user_id=d91fee721beb47bca999f0b5a1deae00)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:51.521174 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bccad71-4d12-4a3c-ad54-19b41f68ab67 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:51.522764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 538/1094] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 07:00:51 2022] GET /identity/v3/users/d91fee721beb47bca999f0b5a1deae00 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:51.528743 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccad71-4d12-4a3c-ad54-19b41f68ab67 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:51.551264 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b236e5-a445-4a9c-9892-ea22115bace0 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:51.558309 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:51.587151 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d494a6be-c41d-4ed1-89de-2f999ce3ff30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:51.587780 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494a6be-c41d-4ed1-89de-2f999ce3ff30 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:51.588058 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494a6be-c41d-4ed1-89de-2f999ce3ff30 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:51.588327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494a6be-c41d-4ed1-89de-2f999ce3ff30 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:51.588994 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d494a6be-c41d-4ed1-89de-2f999ce3ff30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:51.589296 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d494a6be-c41d-4ed1-89de-2f999ce3ff30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:51.590660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d494a6be-c41d-4ed1-89de-2f999ce3ff30 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:51.603191 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 557/1095] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:51.608898 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d494a6be-c41d-4ed1-89de-2f999ce3ff30 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:51.628091 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccad71-4d12-4a3c-ad54-19b41f68ab67 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:51.661667 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887c541-ad29-4d97-a446-2f5da3e5e3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:51.662275 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887c541-ad29-4d97-a446-2f5da3e5e3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:51.662550 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887c541-ad29-4d97-a446-2f5da3e5e3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:51.662818 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887c541-ad29-4d97-a446-2f5da3e5e3b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:51.714257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d494a6be-c41d-4ed1-89de-2f999ce3ff30 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:51.726872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:51.730689 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887c541-ad29-4d97-a446-2f5da3e5e3b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:51.732844 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887c541-ad29-4d97-a446-2f5da3e5e3b2 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:51.748577 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d394d7-675a-493d-8360-91e69b84b08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:51.749129 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d394d7-675a-493d-8360-91e69b84b08f None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:51.749402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d394d7-675a-493d-8360-91e69b84b08f None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:51.749666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d394d7-675a-493d-8360-91e69b84b08f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:51.756744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d394d7-675a-493d-8360-91e69b84b08f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:51.758220 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d394d7-675a-493d-8360-91e69b84b08f None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:51.770714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-30d394d7-675a-493d-8360-91e69b84b08f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:00:51.770714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:51.770714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:51.770714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:51.770714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:51.770714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:51.770714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:51.770714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:51.770714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:51.770714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:51.770714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:51.770714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:51.770714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:00:51.770714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:00:51.770714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:00:51.770714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:00:51.771786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:51.771786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:51.771786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:00:51.771786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:51.771786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:00:51.771786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:00:51.771786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:51.771786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:51.771786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:51.771786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:51.771786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:51.771786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:51.771786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:51.771786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:51.771786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:51.771786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:51.771786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:51.771786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:51.772893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:51.772893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:51.772893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:00:51.772893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:00:51.772893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:00:51.772893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:00:51.772893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:00:51.772893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:00:51.772893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:00:51.772893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:00:51.772893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 558/1096] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Aug 23 07:00:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:00:51.778024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d394d7-675a-493d-8360-91e69b84b08f None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:51.805349 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:51.834435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 539/1097] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:00:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 07:00:51.840517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887c541-ad29-4d97-a446-2f5da3e5e3b2 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:51.880193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d394d7-675a-493d-8360-91e69b84b08f None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:51.915278 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ee73f-c629-44a2-8f34-42a85fc74387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:51.915906 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ee73f-c629-44a2-8f34-42a85fc74387 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:51.916182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ee73f-c629-44a2-8f34-42a85fc74387 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:51.916447 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ee73f-c629-44a2-8f34-42a85fc74387 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:51.917162 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ee73f-c629-44a2-8f34-42a85fc74387 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:51.917466 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ee73f-c629-44a2-8f34-42a85fc74387 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:51.918878 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ee73f-c629-44a2-8f34-42a85fc74387 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:51.933434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:51.937318 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 559/1098] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Aug 23 07:00:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:51.937856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887c541-ad29-4d97-a446-2f5da3e5e3b2 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:51.942970 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ee73f-c629-44a2-8f34-42a85fc74387 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:51.975814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6e643-ac30-4184-ad36-c0dca4da11b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:51.976517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6e643-ac30-4184-ad36-c0dca4da11b1 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:51.976787 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6e643-ac30-4184-ad36-c0dca4da11b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:51.977058 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6e643-ac30-4184-ad36-c0dca4da11b1 None admin] PATH_INFO: `/v3/projects/2fdde1d5ddbd4deda7b56d272054e90f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:51.978168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6e643-ac30-4184-ad36-c0dca4da11b1 None admin] RBAC: Authorizing `identity:get_project(project_id=2fdde1d5ddbd4deda7b56d272054e90f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:51.979645 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6e643-ac30-4184-ad36-c0dca4da11b1 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:51.981271 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 540/1099] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 07:00:51 2022] GET /identity/v3/projects/2fdde1d5ddbd4deda7b56d272054e90f => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:51.987286 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6e643-ac30-4184-ad36-c0dca4da11b1 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:52.031724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:52.048010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ee73f-c629-44a2-8f34-42a85fc74387 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:52.083031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e5de3-2afb-4636-93ec-6336c67cf5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:52.083675 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e5de3-2afb-4636-93ec-6336c67cf5ef None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:52.083953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e5de3-2afb-4636-93ec-6336c67cf5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:52.084224 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e5de3-2afb-4636-93ec-6336c67cf5ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:52.085743 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6e643-ac30-4184-ad36-c0dca4da11b1 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:52.119807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef42b505-382b-4f86-a513-92c7aa847839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:52.120410 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42b505-382b-4f86-a513-92c7aa847839 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:52.120687 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42b505-382b-4f86-a513-92c7aa847839 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:52.120960 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42b505-382b-4f86-a513-92c7aa847839 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:52.121689 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef42b505-382b-4f86-a513-92c7aa847839 None admin] RBAC: Adding query filter params (role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.project.id=2fdde1d5ddbd4deda7b56d272054e90f, user.id=d91fee721beb47bca999f0b5a1deae00) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:52.122005 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef42b505-382b-4f86-a513-92c7aa847839 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:52.123473 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef42b505-382b-4f86-a513-92c7aa847839 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:52.129114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 541/1100] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:00:51 2022] GET /identity/v3/role_assignments?user.id=d91fee721beb47bca999f0b5a1deae00&scope.project.id=2fdde1d5ddbd4deda7b56d272054e90f&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:00:52.152204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e5de3-2afb-4636-93ec-6336c67cf5ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:52.153724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e5de3-2afb-4636-93ec-6336c67cf5ef None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:52.246245 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:52.264196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 560/1101] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:00:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 07:00:52.270136 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef42b505-382b-4f86-a513-92c7aa847839 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:52.304210 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:52.379434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef42b505-382b-4f86-a513-92c7aa847839 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:52.415650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae9e6a-1f54-48f9-a3a9-b43df4fc09bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:52.416278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae9e6a-1f54-48f9-a3a9-b43df4fc09bd None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:52.416552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae9e6a-1f54-48f9-a3a9-b43df4fc09bd None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:52.416817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae9e6a-1f54-48f9-a3a9-b43df4fc09bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:52.427651 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae9e6a-1f54-48f9-a3a9-b43df4fc09bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:52.429100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae9e6a-1f54-48f9-a3a9-b43df4fc09bd None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:52.443149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-5fae9e6a-1f54-48f9-a3a9-b43df4fc09bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:52.443149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:52.443149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:52.443149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:52.443149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:52.443149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:52.443149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:52.443149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:52.443149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:52.443149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:52.443149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:52.443149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:52.443149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:00:52.443149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:00:52.443149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:00:52.443149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:00:52.444144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:52.444144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:52.444144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:52.444144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:52.444144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:52.444144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:52.444144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:52.444144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:52.444144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:52.444144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:52.444144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:52.444144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:52.444144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:52.444144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:52.444144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:52.444144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:52.444144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:00:52.445039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:00:52.445039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:00:52.445039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:52.445039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 542/1102] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:52.450314 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e5de3-2afb-4636-93ec-6336c67cf5ef None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:52.551719 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e5de3-2afb-4636-93ec-6336c67cf5ef None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:52.589017 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9d83f-ebd9-47dc-877b-301d06b2c035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:52.589586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9d83f-ebd9-47dc-877b-301d06b2c035 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:52.589857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9d83f-ebd9-47dc-877b-301d06b2c035 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:52.590122 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9d83f-ebd9-47dc-877b-301d06b2c035 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:52.590761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9d83f-ebd9-47dc-877b-301d06b2c035 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:52.591060 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9d83f-ebd9-47dc-877b-301d06b2c035 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:52.592451 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9d83f-ebd9-47dc-877b-301d06b2c035 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:52.604582 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 561/1103] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:52.610461 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae9e6a-1f54-48f9-a3a9-b43df4fc09bd None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:52.700653 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:52.719752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae9e6a-1f54-48f9-a3a9-b43df4fc09bd None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:52.752919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21ac66-3ed6-481e-aac6-fc752002f352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:52.753546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21ac66-3ed6-481e-aac6-fc752002f352 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:52.753824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21ac66-3ed6-481e-aac6-fc752002f352 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:52.754094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21ac66-3ed6-481e-aac6-fc752002f352 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:52.760948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf21ac66-3ed6-481e-aac6-fc752002f352 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:52.762411 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf21ac66-3ed6-481e-aac6-fc752002f352 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:52.773575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-bf21ac66-3ed6-481e-aac6-fc752002f352 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:52.773575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:00:52.773575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:00:52.773575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:00:52.773575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:00:52.773575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:00:52.773575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:00:52.773575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:00:52.773575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:00:52.773575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:00:52.773575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:00:52.773575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:00:52.773575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:00:52.773575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:00:52.773575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:00:52.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:00:52.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:00:52.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:00:52.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:00:52.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:00:52.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:00:52.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:00:52.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:00:52.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:00:52.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:00:52.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:00:52.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:00:52.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:00:52.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:00:52.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:00:52.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:00:52.774504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:00:52.775710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:00:52.775710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:00:52.775710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:00:52.775710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:00:52.775710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:00:52.775710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:00:52.775710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:00:52.775710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:00:52.775710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 543/1104] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:00:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:00:52.780772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9d83f-ebd9-47dc-877b-301d06b2c035 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:52.830142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:52.885388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9d83f-ebd9-47dc-877b-301d06b2c035 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:52.919215 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0610877-beb7-440c-9216-09479779728a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:52.919835 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0610877-beb7-440c-9216-09479779728a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:52.920112 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0610877-beb7-440c-9216-09479779728a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:52.920345 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0610877-beb7-440c-9216-09479779728a None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:52.921056 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0610877-beb7-440c-9216-09479779728a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:52.921353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0610877-beb7-440c-9216-09479779728a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:52.922774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0610877-beb7-440c-9216-09479779728a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:52.934714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 562/1105] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:00:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:52.941215 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21ac66-3ed6-481e-aac6-fc752002f352 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:53.051435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21ac66-3ed6-481e-aac6-fc752002f352 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:53.085338 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbbaba-0974-4501-927d-3291d0e349c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:53.085976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbbaba-0974-4501-927d-3291d0e349c3 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:53.086256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbbaba-0974-4501-927d-3291d0e349c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:53.086523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbbaba-0974-4501-927d-3291d0e349c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:53.087266 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fbbaba-0974-4501-927d-3291d0e349c3 None admin] RBAC: Adding query filter params (role.id=6d1af19135484d85af7accf2fdeb3cab, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=a8d2cd93042c4583b63afe629d47fce0) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:53.087608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fbbaba-0974-4501-927d-3291d0e349c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:53.089052 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fbbaba-0974-4501-927d-3291d0e349c3 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:53.095070 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 544/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:00:52 2022] GET /identity/v3/role_assignments?user.id=a8d2cd93042c4583b63afe629d47fce0&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=6d1af19135484d85af7accf2fdeb3cab => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:54.049151 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54895f12-73f2-4aa0-b537-c4db70d503c5 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:54.049949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54895f12-73f2-4aa0-b537-c4db70d503c5 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:54.049949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54895f12-73f2-4aa0-b537-c4db70d503c5 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:54.052113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 563/1107] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:54.057615 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad26ab1-dbd5-493f-be6d-cefdf37de652 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:54.058053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad26ab1-dbd5-493f-be6d-cefdf37de652 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:54.058147 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad26ab1-dbd5-493f-be6d-cefdf37de652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:54.072945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:54.078949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-dad26ab1-dbd5-493f-be6d-cefdf37de652 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:54.081071 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-dad26ab1-dbd5-493f-be6d-cefdf37de652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:54.154909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 545/1108] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:54.426797 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54895f12-73f2-4aa0-b537-c4db70d503c5 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:54.539302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54895f12-73f2-4aa0-b537-c4db70d503c5 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:54.541763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:54.576168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a9319-d476-422f-8158-97b54b7a9eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:54.576817 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a9319-d476-422f-8158-97b54b7a9eac None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:54.577092 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a9319-d476-422f-8158-97b54b7a9eac None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:54.577364 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a9319-d476-422f-8158-97b54b7a9eac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:54.578492 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a9319-d476-422f-8158-97b54b7a9eac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:54.579980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a9319-d476-422f-8158-97b54b7a9eac None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:54.581562 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 564/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:00:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:54.591095 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad26ab1-dbd5-493f-be6d-cefdf37de652 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:54.702434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad26ab1-dbd5-493f-be6d-cefdf37de652 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:54.737949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba388510-6b9e-4e43-a77a-392d70f8eccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:54.738554 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba388510-6b9e-4e43-a77a-392d70f8eccb None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:54.738831 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba388510-6b9e-4e43-a77a-392d70f8eccb None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:54.739105 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba388510-6b9e-4e43-a77a-392d70f8eccb None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:54.739784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba388510-6b9e-4e43-a77a-392d70f8eccb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:54.741232 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba388510-6b9e-4e43-a77a-392d70f8eccb None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:54.750619 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:54.766842 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 546/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 23 07:00:54 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:00:56.684474 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46765c93-0b24-40c3-a8d7-329546c5f570 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:56.684956 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46765c93-0b24-40c3-a8d7-329546c5f570 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:56.685075 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46765c93-0b24-40c3-a8d7-329546c5f570 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:56.686365 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 565/1111] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:00:56.693082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ef32f-f453-43c5-9234-7c296d7397d4 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:56.693504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ef32f-f453-43c5-9234-7c296d7397d4 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:56.693614 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ef32f-f453-43c5-9234-7c296d7397d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:56.715366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-af8ef32f-f453-43c5-9234-7c296d7397d4 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:00:56.716609 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-af8ef32f-f453-43c5-9234-7c296d7397d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:00:56.793116 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 547/1112] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:00:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:00:57.067288 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46765c93-0b24-40c3-a8d7-329546c5f570 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:57.173789 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46765c93-0b24-40c3-a8d7-329546c5f570 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:57.209586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90455f-0a64-4dee-9995-0c62963e84b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:57.210193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90455f-0a64-4dee-9995-0c62963e84b6 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:57.210465 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90455f-0a64-4dee-9995-0c62963e84b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:57.210736 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90455f-0a64-4dee-9995-0c62963e84b6 None admin] PATH_INFO: `/v3/roles/c0ef5f54fa5e4cbb93cb48c9c02b83b7` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:57.212798 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90455f-0a64-4dee-9995-0c62963e84b6 None admin] RBAC: Authorizing `identity:get_role(role_id=c0ef5f54fa5e4cbb93cb48c9c02b83b7)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:57.214492 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90455f-0a64-4dee-9995-0c62963e84b6 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:57.215846 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 566/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 07:00:57 2022] GET /identity/v3/roles/c0ef5f54fa5e4cbb93cb48c9c02b83b7 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:57.222641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ef32f-f453-43c5-9234-7c296d7397d4 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:57.340055 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ef32f-f453-43c5-9234-7c296d7397d4 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:57.364327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:57.377834 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4111bf2-d851-4155-ab13-68256a90f1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:57.378476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4111bf2-d851-4155-ab13-68256a90f1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:57.378753 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4111bf2-d851-4155-ab13-68256a90f1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:57.379026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4111bf2-d851-4155-ab13-68256a90f1c1 None admin] PATH_INFO: `/v3/users/2ab47a797b2047cfaea8d4181ad3eda2` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:57.393378 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4111bf2-d851-4155-ab13-68256a90f1c1 None admin] RBAC: Authorizing `identity:get_user(user_id=2ab47a797b2047cfaea8d4181ad3eda2)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:57.394865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4111bf2-d851-4155-ab13-68256a90f1c1 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:57.396583 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 548/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 07:00:57 2022] GET /identity/v3/users/2ab47a797b2047cfaea8d4181ad3eda2 => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:57.402700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90455f-0a64-4dee-9995-0c62963e84b6 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:57.447696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:57.512916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90455f-0a64-4dee-9995-0c62963e84b6 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:57.547098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0d327-afd3-419b-a931-209e03e7d147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:57.547711 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0d327-afd3-419b-a931-209e03e7d147 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:57.547985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0d327-afd3-419b-a931-209e03e7d147 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:57.548251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0d327-afd3-419b-a931-209e03e7d147 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:57.617265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c0d327-afd3-419b-a931-209e03e7d147 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:57.618659 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c0d327-afd3-419b-a931-209e03e7d147 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:57.659198 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:00:57.724124 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 567/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:00:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:00:57.732270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4111bf2-d851-4155-ab13-68256a90f1c1 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:57.851950 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4111bf2-d851-4155-ab13-68256a90f1c1 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:57.895617 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf78c8-3241-4658-98fe-70382cf0895c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:57.896491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf78c8-3241-4658-98fe-70382cf0895c None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:57.896844 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf78c8-3241-4658-98fe-70382cf0895c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:57.897189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf78c8-3241-4658-98fe-70382cf0895c None admin] PATH_INFO: `/v3/projects/2fdde1d5ddbd4deda7b56d272054e90f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:57.898953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf78c8-3241-4658-98fe-70382cf0895c None admin] RBAC: Authorizing `identity:get_project(project_id=2fdde1d5ddbd4deda7b56d272054e90f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:57.900548 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf78c8-3241-4658-98fe-70382cf0895c None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:57.902169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 549/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 07:00:57 2022] GET /identity/v3/projects/2fdde1d5ddbd4deda7b56d272054e90f => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:00:57.909128 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0d327-afd3-419b-a931-209e03e7d147 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:00:58.023975 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0d327-afd3-419b-a931-209e03e7d147 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:00:58.059718 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf430bdb-f242-44ee-b6c7-f677449f16fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:00:58.060305 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf430bdb-f242-44ee-b6c7-f677449f16fb None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:00:58.060575 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf430bdb-f242-44ee-b6c7-f677449f16fb None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:00:58.060841 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf430bdb-f242-44ee-b6c7-f677449f16fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:00:58.061552 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf430bdb-f242-44ee-b6c7-f677449f16fb None admin] RBAC: Adding query filter params (role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7, scope.project.id=2fdde1d5ddbd4deda7b56d272054e90f, user.id=2ab47a797b2047cfaea8d4181ad3eda2) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:00:58.061854 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf430bdb-f242-44ee-b6c7-f677449f16fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:00:58.063270 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf430bdb-f242-44ee-b6c7-f677449f16fb None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:00:58.068749 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 568/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:00:57 2022] GET /identity/v3/role_assignments?user.id=2ab47a797b2047cfaea8d4181ad3eda2&scope.project.id=2fdde1d5ddbd4deda7b56d272054e90f&role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:00.020967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6189450-70b9-40b8-a2f7-ca6ff0963769 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:00.021548 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6189450-70b9-40b8-a2f7-ca6ff0963769 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:00.021548 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6189450-70b9-40b8-a2f7-ca6ff0963769 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:00.022743 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 550/1118] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:01:00.029017 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee586b-0eb1-43bb-acdc-8c2b13214a55 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:00.029746 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee586b-0eb1-43bb-acdc-8c2b13214a55 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:00.029981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee586b-0eb1-43bb-acdc-8c2b13214a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:00.042118 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:00.058547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-4fee586b-0eb1-43bb-acdc-8c2b13214a55 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:01:00.059817 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4fee586b-0eb1-43bb-acdc-8c2b13214a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:01:00.136614 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 569/1119] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:01:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:01:00.404987 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6189450-70b9-40b8-a2f7-ca6ff0963769 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:00.481301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:00.546493 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6189450-70b9-40b8-a2f7-ca6ff0963769 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:00.584535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02240757-10f1-4487-b99e-8721d83c74cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:00.585152 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02240757-10f1-4487-b99e-8721d83c74cf None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:00.585422 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02240757-10f1-4487-b99e-8721d83c74cf None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:00.585687 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02240757-10f1-4487-b99e-8721d83c74cf None admin] PATH_INFO: `/v3/roles/c0ef5f54fa5e4cbb93cb48c9c02b83b7` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:00.587192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02240757-10f1-4487-b99e-8721d83c74cf None admin] RBAC: Authorizing `identity:get_role(role_id=c0ef5f54fa5e4cbb93cb48c9c02b83b7)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:00.588613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02240757-10f1-4487-b99e-8721d83c74cf None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:00.589957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 551/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 07:01:00 2022] GET /identity/v3/roles/c0ef5f54fa5e4cbb93cb48c9c02b83b7 => generated 223 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:00.599228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee586b-0eb1-43bb-acdc-8c2b13214a55 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:00.701622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee586b-0eb1-43bb-acdc-8c2b13214a55 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:00.743626 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a956d4b-2210-453b-89b6-d82b37e1c693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:00.744251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a956d4b-2210-453b-89b6-d82b37e1c693 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:00.744533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a956d4b-2210-453b-89b6-d82b37e1c693 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:00.744803 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a956d4b-2210-453b-89b6-d82b37e1c693 None admin] PATH_INFO: `/v3/users/2ab47a797b2047cfaea8d4181ad3eda2` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:00.745947 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a956d4b-2210-453b-89b6-d82b37e1c693 None admin] RBAC: Authorizing `identity:get_user(user_id=2ab47a797b2047cfaea8d4181ad3eda2)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:00.747397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a956d4b-2210-453b-89b6-d82b37e1c693 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:00.749042 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 570/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 07:01:00 2022] GET /identity/v3/users/2ab47a797b2047cfaea8d4181ad3eda2 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:00.755851 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02240757-10f1-4487-b99e-8721d83c74cf None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:00.885782 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02240757-10f1-4487-b99e-8721d83c74cf None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:00.902262 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:00.921948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74cef48-cabc-4bc0-a0bc-59cacf0d2f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:00.922495 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cef48-cabc-4bc0-a0bc-59cacf0d2f21 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:00.922770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cef48-cabc-4bc0-a0bc-59cacf0d2f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:00.923040 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cef48-cabc-4bc0-a0bc-59cacf0d2f21 None admin] PATH_INFO: `/v3/projects/2fdde1d5ddbd4deda7b56d272054e90f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:00.924266 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74cef48-cabc-4bc0-a0bc-59cacf0d2f21 None admin] RBAC: Authorizing `identity:get_project(project_id=2fdde1d5ddbd4deda7b56d272054e90f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:00.925712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74cef48-cabc-4bc0-a0bc-59cacf0d2f21 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:00.927345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 552/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 07:01:00 2022] GET /identity/v3/projects/2fdde1d5ddbd4deda7b56d272054e90f => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:00.934502 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a956d4b-2210-453b-89b6-d82b37e1c693 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:00.937590 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:01.056661 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a956d4b-2210-453b-89b6-d82b37e1c693 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:01.092594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378d3b7-bdc1-496b-b368-93b3e6eb3246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:01.093237 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378d3b7-bdc1-496b-b368-93b3e6eb3246 None admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:01.093517 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378d3b7-bdc1-496b-b368-93b3e6eb3246 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:01.093788 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378d3b7-bdc1-496b-b368-93b3e6eb3246 None admin] PATH_INFO: `/v3/projects/2fdde1d5ddbd4deda7b56d272054e90f/users/2ab47a797b2047cfaea8d4181ad3eda2/roles/c0ef5f54fa5e4cbb93cb48c9c02b83b7` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:01.096124 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6378d3b7-bdc1-496b-b368-93b3e6eb3246 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fdde1d5ddbd4deda7b56d272054e90f, user_id=2ab47a797b2047cfaea8d4181ad3eda2, role_id=c0ef5f54fa5e4cbb93cb48c9c02b83b7)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:01.097576 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6378d3b7-bdc1-496b-b368-93b3e6eb3246 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:01.114040 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 571/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 07:01:00 2022] PUT /identity/v3/projects/2fdde1d5ddbd4deda7b56d272054e90f/users/2ab47a797b2047cfaea8d4181ad3eda2/roles/c0ef5f54fa5e4cbb93cb48c9c02b83b7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:01:02.979603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a6176-8d29-4698-8d25-e6dc6ca31c32 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:02.980405 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a6176-8d29-4698-8d25-e6dc6ca31c32 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:02.980405 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a6176-8d29-4698-8d25-e6dc6ca31c32 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:02.982592 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 553/1124] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:01:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:01:02.988994 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b797b5-e996-492a-b60c-28e888406dc6 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:02.989451 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b797b5-e996-492a-b60c-28e888406dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:02.989544 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b797b5-e996-492a-b60c-28e888406dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:03.010293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-80b797b5-e996-492a-b60c-28e888406dc6 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:01:03.011526 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-80b797b5-e996-492a-b60c-28e888406dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:01:03.075576 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:03.088956 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 572/1125] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:01:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:01:03.353659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a6176-8d29-4698-8d25-e6dc6ca31c32 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:03.472905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a6176-8d29-4698-8d25-e6dc6ca31c32 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:03.508659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47028c5-57e8-4040-85ff-617ef5209175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:03.509277 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47028c5-57e8-4040-85ff-617ef5209175 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:03.509559 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47028c5-57e8-4040-85ff-617ef5209175 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:03.509830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47028c5-57e8-4040-85ff-617ef5209175 None admin] PATH_INFO: `/v3/roles/c0ef5f54fa5e4cbb93cb48c9c02b83b7` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:03.511365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47028c5-57e8-4040-85ff-617ef5209175 None admin] RBAC: Authorizing `identity:get_role(role_id=c0ef5f54fa5e4cbb93cb48c9c02b83b7)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:03.512784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47028c5-57e8-4040-85ff-617ef5209175 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:03.514112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 554/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 07:01:03 2022] GET /identity/v3/roles/c0ef5f54fa5e4cbb93cb48c9c02b83b7 => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:03.520435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b797b5-e996-492a-b60c-28e888406dc6 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:03.633643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b797b5-e996-492a-b60c-28e888406dc6 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:03.675278 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795e3fc-a725-40f0-bc69-cd625bac8e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:03.675956 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795e3fc-a725-40f0-bc69-cd625bac8e23 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:03.676237 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795e3fc-a725-40f0-bc69-cd625bac8e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:03.676512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795e3fc-a725-40f0-bc69-cd625bac8e23 None admin] PATH_INFO: `/v3/users/2ab47a797b2047cfaea8d4181ad3eda2` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:03.677702 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795e3fc-a725-40f0-bc69-cd625bac8e23 None admin] RBAC: Authorizing `identity:get_user(user_id=2ab47a797b2047cfaea8d4181ad3eda2)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:03.679151 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795e3fc-a725-40f0-bc69-cd625bac8e23 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:03.680876 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 573/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 07:01:03 2022] GET /identity/v3/users/2ab47a797b2047cfaea8d4181ad3eda2 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:03.686823 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47028c5-57e8-4040-85ff-617ef5209175 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:03.741541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:03.791581 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47028c5-57e8-4040-85ff-617ef5209175 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:03.828726 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57aca9-79d5-4504-bc73-5c954cef6189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:03.829337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57aca9-79d5-4504-bc73-5c954cef6189 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:03.829614 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57aca9-79d5-4504-bc73-5c954cef6189 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:03.829885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57aca9-79d5-4504-bc73-5c954cef6189 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:03.899760 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc57aca9-79d5-4504-bc73-5c954cef6189 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:03.901149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc57aca9-79d5-4504-bc73-5c954cef6189 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:03.952503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:04.002904 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 555/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:01:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:01:04.009340 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795e3fc-a725-40f0-bc69-cd625bac8e23 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:04.087301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:04.136016 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795e3fc-a725-40f0-bc69-cd625bac8e23 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:04.173843 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a19503-63da-4a2f-a536-da726699f5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:04.174397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a19503-63da-4a2f-a536-da726699f5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:04.174670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a19503-63da-4a2f-a536-da726699f5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:04.174941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a19503-63da-4a2f-a536-da726699f5b9 None admin] PATH_INFO: `/v3/projects/2fdde1d5ddbd4deda7b56d272054e90f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:04.176218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a19503-63da-4a2f-a536-da726699f5b9 None admin] RBAC: Authorizing `identity:get_project(project_id=2fdde1d5ddbd4deda7b56d272054e90f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:04.177659 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a19503-63da-4a2f-a536-da726699f5b9 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:04.179290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 574/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 07:01:04 2022] GET /identity/v3/projects/2fdde1d5ddbd4deda7b56d272054e90f => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:04.185912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57aca9-79d5-4504-bc73-5c954cef6189 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:04.297246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57aca9-79d5-4504-bc73-5c954cef6189 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:04.333469 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f064bcf-75f0-4e2d-a4c0-09b79428e52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:04.334076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f064bcf-75f0-4e2d-a4c0-09b79428e52f None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:04.334347 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f064bcf-75f0-4e2d-a4c0-09b79428e52f None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:04.334616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f064bcf-75f0-4e2d-a4c0-09b79428e52f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:04.335350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f064bcf-75f0-4e2d-a4c0-09b79428e52f None admin] RBAC: Adding query filter params (role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7, scope.project.id=2fdde1d5ddbd4deda7b56d272054e90f, user.id=2ab47a797b2047cfaea8d4181ad3eda2) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:01:04.335684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f064bcf-75f0-4e2d-a4c0-09b79428e52f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:04.337174 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f064bcf-75f0-4e2d-a4c0-09b79428e52f None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:04.343088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 556/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:01:04 2022] GET /identity/v3/role_assignments?user.id=2ab47a797b2047cfaea8d4181ad3eda2&scope.project.id=2fdde1d5ddbd4deda7b56d272054e90f&role.id=c0ef5f54fa5e4cbb93cb48c9c02b83b7 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:06.247223 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a4608-d348-451c-9081-0d45c982e31b None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:06.247824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a4608-d348-451c-9081-0d45c982e31b None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:06.247824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a4608-d348-451c-9081-0d45c982e31b None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:06.249127 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 575/1131] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:01:06.255685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37589544-2897-4ea9-b429-14da0109b36b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:06.256337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37589544-2897-4ea9-b429-14da0109b36b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:06.256530 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37589544-2897-4ea9-b429-14da0109b36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:06.281981 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-37589544-2897-4ea9-b429-14da0109b36b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:01:06.283186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-37589544-2897-4ea9-b429-14da0109b36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:01:06.294403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:06.361880 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 557/1132] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:01:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:01:06.628176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a4608-d348-451c-9081-0d45c982e31b None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:06.741144 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a4608-d348-451c-9081-0d45c982e31b None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:06.768649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:06.778806 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2584c6-6209-4cad-923e-9b552514ab35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:06.779458 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2584c6-6209-4cad-923e-9b552514ab35 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:06.779744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2584c6-6209-4cad-923e-9b552514ab35 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:06.780020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2584c6-6209-4cad-923e-9b552514ab35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:06.781244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2584c6-6209-4cad-923e-9b552514ab35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:06.782705 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2584c6-6209-4cad-923e-9b552514ab35 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:06.784335 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 576/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:01:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:06.790523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37589544-2897-4ea9-b429-14da0109b36b None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:06.908385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37589544-2897-4ea9-b429-14da0109b36b None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:06.944224 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07179b-9db0-4451-9bfe-2aac5f203426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:06.944829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07179b-9db0-4451-9bfe-2aac5f203426 None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:06.945149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07179b-9db0-4451-9bfe-2aac5f203426 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:06.945486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07179b-9db0-4451-9bfe-2aac5f203426 None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:06.946352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07179b-9db0-4451-9bfe-2aac5f203426 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:06.947936 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07179b-9db0-4451-9bfe-2aac5f203426 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:06.972316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:06.983522 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 558/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 23 07:01:06 2022] POST /identity/v3/projects => generated 311 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:01:08.884042 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daec88-001f-4e25-8d55-b3cd0c3c79bc None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:08.884874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daec88-001f-4e25-8d55-b3cd0c3c79bc None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:08.884874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daec88-001f-4e25-8d55-b3cd0c3c79bc None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:08.887095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 577/1135] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:01:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:01:08.893569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836c72f-12c0-4631-a9c3-7e2f156d587d None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:08.894279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836c72f-12c0-4631-a9c3-7e2f156d587d None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:08.894434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836c72f-12c0-4631-a9c3-7e2f156d587d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:08.920820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-8836c72f-12c0-4631-a9c3-7e2f156d587d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:01:08.921932 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8836c72f-12c0-4631-a9c3-7e2f156d587d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:01:08.995347 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 559/1136] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:01:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:01:09.272566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daec88-001f-4e25-8d55-b3cd0c3c79bc None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:09.385419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daec88-001f-4e25-8d55-b3cd0c3c79bc None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:09.424706 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e22f0c-c694-4e5a-99a0-b00333fdc53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:09.425313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e22f0c-c694-4e5a-99a0-b00333fdc53b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:09.425588 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e22f0c-c694-4e5a-99a0-b00333fdc53b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:09.425863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e22f0c-c694-4e5a-99a0-b00333fdc53b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:09.426962 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e22f0c-c694-4e5a-99a0-b00333fdc53b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:09.428425 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e22f0c-c694-4e5a-99a0-b00333fdc53b None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:09.429984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 578/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:01:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:09.436921 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836c72f-12c0-4631-a9c3-7e2f156d587d None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:09.542734 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836c72f-12c0-4631-a9c3-7e2f156d587d None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:09.569366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:09.578297 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caeb588-676c-405f-89e2-5d680b4d87c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:09.578896 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caeb588-676c-405f-89e2-5d680b4d87c9 None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:09.579196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caeb588-676c-405f-89e2-5d680b4d87c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:09.579493 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caeb588-676c-405f-89e2-5d680b4d87c9 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:09.580125 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caeb588-676c-405f-89e2-5d680b4d87c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:09.581567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caeb588-676c-405f-89e2-5d680b4d87c9 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:09.604024 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 560/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 23 07:01:09 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:01:11.531973 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b3ddb-6631-40ee-a4ac-6eb3c9a0156c None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:11.532560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b3ddb-6631-40ee-a4ac-6eb3c9a0156c None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:11.532560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b3ddb-6631-40ee-a4ac-6eb3c9a0156c None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:11.533873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 579/1139] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:01:11.540201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d15b4-1300-4808-9840-a36f77188869 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:11.540910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d15b4-1300-4808-9840-a36f77188869 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:11.541057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d15b4-1300-4808-9840-a36f77188869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:11.565423 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-049d15b4-1300-4808-9840-a36f77188869 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:01:11.566587 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-049d15b4-1300-4808-9840-a36f77188869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:01:11.646697 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 561/1140] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:01:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:01:11.915701 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b3ddb-6631-40ee-a4ac-6eb3c9a0156c None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:11.992686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:12.045709 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b3ddb-6631-40ee-a4ac-6eb3c9a0156c None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:12.085588 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9fff1-735d-4445-82f2-bdfb907cb3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:12.086193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9fff1-735d-4445-82f2-bdfb907cb3de None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:12.086468 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9fff1-735d-4445-82f2-bdfb907cb3de None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:12.086747 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9fff1-735d-4445-82f2-bdfb907cb3de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:12.097884 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9fff1-735d-4445-82f2-bdfb907cb3de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:01:12.103290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-2ac9fff1-735d-4445-82f2-bdfb907cb3de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:01:12.103290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:01:12.103290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:01:12.103290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:01:12.103290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:01:12.103290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:01:12.103290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:01:12.103290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:01:12.103290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:01:12.103290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:01:12.103290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:01:12.103290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:01:12.103290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:01:12.103290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:01:12.103290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:01:12.103290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:01:12.107439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:01:12.107439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:01:12.107439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:01:12.107439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:01:12.107439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:01:12.107439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:01:12.107439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:01:12.107439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:01:12.107439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:01:12.107439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:01:12.107439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:01:12.107439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:01:12.107439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:01:12.107439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:01:12.107439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:01:12.107439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:01:12.108555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:01:12.108555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:01:12.108555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:01:12.108555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:01:12.108555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:01:12.108555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:01:12.108555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:01:12.108555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:01:12.108555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:01:12.108555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:01:12.108555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:01:12.108555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:01:12.108555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 580/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:01:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:01:12.110621 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d15b4-1300-4808-9840-a36f77188869 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:12.213535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d15b4-1300-4808-9840-a36f77188869 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:12.227442 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:12.249894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d6604-2e66-4d21-b6a1-881a28826a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:12.250524 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d6604-2e66-4d21-b6a1-881a28826a33 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:12.250798 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d6604-2e66-4d21-b6a1-881a28826a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:12.251060 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d6604-2e66-4d21-b6a1-881a28826a33 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:12.251758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d6604-2e66-4d21-b6a1-881a28826a33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:01:12.252058 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d6604-2e66-4d21-b6a1-881a28826a33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:12.253436 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d6604-2e66-4d21-b6a1-881a28826a33 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:12.261779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 562/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 07:01:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:12.270283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9fff1-735d-4445-82f2-bdfb907cb3de None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:12.380505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9fff1-735d-4445-82f2-bdfb907cb3de None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:12.405325 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:12.420237 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74a3e3-eec6-4841-933a-f54d4e740bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:12.420888 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74a3e3-eec6-4841-933a-f54d4e740bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:12.421159 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74a3e3-eec6-4841-933a-f54d4e740bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:12.421426 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74a3e3-eec6-4841-933a-f54d4e740bd2 None admin] PATH_INFO: `/v3/users/b1a27cd7cc0d412790057890164e9bb6` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:12.435982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74a3e3-eec6-4841-933a-f54d4e740bd2 None admin] RBAC: Authorizing `identity:get_user(user_id=b1a27cd7cc0d412790057890164e9bb6)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:12.437446 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74a3e3-eec6-4841-933a-f54d4e740bd2 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:12.439035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 581/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 07:01:12 2022] GET /identity/v3/users/b1a27cd7cc0d412790057890164e9bb6 => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:12.445117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d6604-2e66-4d21-b6a1-881a28826a33 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:12.560455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d6604-2e66-4d21-b6a1-881a28826a33 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:12.598685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755659df-6ba1-4143-8e10-98f5f9a32c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:12.599289 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755659df-6ba1-4143-8e10-98f5f9a32c73 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:12.599599 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755659df-6ba1-4143-8e10-98f5f9a32c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:12.599872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755659df-6ba1-4143-8e10-98f5f9a32c73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:12.647149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:12.676196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755659df-6ba1-4143-8e10-98f5f9a32c73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:12.677669 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755659df-6ba1-4143-8e10-98f5f9a32c73 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:12.786205 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 563/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:01:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:01:12.792495 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74a3e3-eec6-4841-933a-f54d4e740bd2 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:12.897370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74a3e3-eec6-4841-933a-f54d4e740bd2 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:12.934128 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5808da8-2c05-45c2-98d4-6bb213520248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:12.934745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5808da8-2c05-45c2-98d4-6bb213520248 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:12.935022 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5808da8-2c05-45c2-98d4-6bb213520248 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:12.935287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5808da8-2c05-45c2-98d4-6bb213520248 None admin] PATH_INFO: `/v3/projects/6942c63862aa4a4996be8098c9e5b569` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:12.936555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5808da8-2c05-45c2-98d4-6bb213520248 None admin] RBAC: Authorizing `identity:get_project(project_id=6942c63862aa4a4996be8098c9e5b569)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:12.938008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5808da8-2c05-45c2-98d4-6bb213520248 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:12.939643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 582/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 07:01:12 2022] GET /identity/v3/projects/6942c63862aa4a4996be8098c9e5b569 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:12.945908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755659df-6ba1-4143-8e10-98f5f9a32c73 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:13.026662 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:13.051662 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755659df-6ba1-4143-8e10-98f5f9a32c73 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:13.089772 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0161db4-d072-41f7-a66b-76ffa8fba635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:13.090361 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0161db4-d072-41f7-a66b-76ffa8fba635 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:13.090639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0161db4-d072-41f7-a66b-76ffa8fba635 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:13.090908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0161db4-d072-41f7-a66b-76ffa8fba635 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:13.091658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0161db4-d072-41f7-a66b-76ffa8fba635 None admin] RBAC: Adding query filter params (role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.project.id=6942c63862aa4a4996be8098c9e5b569, user.id=b1a27cd7cc0d412790057890164e9bb6) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:01:13.091965 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0161db4-d072-41f7-a66b-76ffa8fba635 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:13.093413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0161db4-d072-41f7-a66b-76ffa8fba635 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:13.098865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 564/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:01:12 2022] GET /identity/v3/role_assignments?user.id=b1a27cd7cc0d412790057890164e9bb6&scope.project.id=6942c63862aa4a4996be8098c9e5b569&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:14.981719 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd064e74-496f-483b-bbdf-dd3f7138c4eb None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:14.982548 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd064e74-496f-483b-bbdf-dd3f7138c4eb None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:14.982548 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd064e74-496f-483b-bbdf-dd3f7138c4eb None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:14.984643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 583/1147] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:01:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:01:14.993367 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61134d-fe00-412c-bfa3-a43273e6d85d None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:14.993756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61134d-fe00-412c-bfa3-a43273e6d85d None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:14.994006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61134d-fe00-412c-bfa3-a43273e6d85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:15.014379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-8d61134d-fe00-412c-bfa3-a43273e6d85d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:01:15.015584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8d61134d-fe00-412c-bfa3-a43273e6d85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:01:15.088698 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 565/1148] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:01:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:01:15.371655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd064e74-496f-483b-bbdf-dd3f7138c4eb None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:15.430366 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:15.489057 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd064e74-496f-483b-bbdf-dd3f7138c4eb None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:15.525909 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280034ed-1d1a-4954-ae9d-af4f915b0f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:15.526529 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280034ed-1d1a-4954-ae9d-af4f915b0f18 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:15.526807 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280034ed-1d1a-4954-ae9d-af4f915b0f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:15.527070 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280034ed-1d1a-4954-ae9d-af4f915b0f18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:15.537844 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280034ed-1d1a-4954-ae9d-af4f915b0f18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:01:15.543137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-280034ed-1d1a-4954-ae9d-af4f915b0f18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:01:15.543137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:01:15.543137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:01:15.543137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:01:15.543137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:01:15.543137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:01:15.543137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:01:15.543137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:01:15.543137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:01:15.543137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:01:15.543137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:01:15.543137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:01:15.543137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:01:15.543137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:01:15.543137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:01:15.543137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:01:15.544221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:01:15.544221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:01:15.544221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:01:15.544221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:01:15.544221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:01:15.544221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:01:15.544221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:01:15.544221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:01:15.544221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:01:15.544221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:01:15.544221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:01:15.544221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:01:15.544221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:01:15.544221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:01:15.544221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:01:15.544221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:01:15.545243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:01:15.545243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:01:15.545243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:01:15.545243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:01:15.545243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:01:15.545243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:01:15.545243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:01:15.545243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:01:15.545243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:01:15.545243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:01:15.545243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:01:15.545243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:01:15.545243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 584/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:01:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:01:15.551091 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61134d-fe00-412c-bfa3-a43273e6d85d None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:15.631259 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:15.668778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61134d-fe00-412c-bfa3-a43273e6d85d None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:15.704336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b3797-bb6f-4a4b-8fa2-95621f954912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:15.704941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b3797-bb6f-4a4b-8fa2-95621f954912 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:15.705219 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b3797-bb6f-4a4b-8fa2-95621f954912 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:15.705489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b3797-bb6f-4a4b-8fa2-95621f954912 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:15.706152 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b3797-bb6f-4a4b-8fa2-95621f954912 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:01:15.706456 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b3797-bb6f-4a4b-8fa2-95621f954912 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:15.707892 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b3797-bb6f-4a4b-8fa2-95621f954912 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:15.716957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 566/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 07:01:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:15.723245 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280034ed-1d1a-4954-ae9d-af4f915b0f18 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:15.831189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280034ed-1d1a-4954-ae9d-af4f915b0f18 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:15.842478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:15.876753 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981349d3-2935-4704-963a-08bc8b56a7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:15.877374 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981349d3-2935-4704-963a-08bc8b56a7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:15.877647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981349d3-2935-4704-963a-08bc8b56a7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:15.877902 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981349d3-2935-4704-963a-08bc8b56a7e7 None admin] PATH_INFO: `/v3/users/b1a27cd7cc0d412790057890164e9bb6` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:15.879199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981349d3-2935-4704-963a-08bc8b56a7e7 None admin] RBAC: Authorizing `identity:get_user(user_id=b1a27cd7cc0d412790057890164e9bb6)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:15.880689 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981349d3-2935-4704-963a-08bc8b56a7e7 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:15.882332 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 585/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 07:01:15 2022] GET /identity/v3/users/b1a27cd7cc0d412790057890164e9bb6 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:15.890038 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b3797-bb6f-4a4b-8fa2-95621f954912 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:16.008539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b3797-bb6f-4a4b-8fa2-95621f954912 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:16.039733 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:16.046116 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca65fb-d6f6-4cba-bb7b-a5a1b2673dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:16.046733 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca65fb-d6f6-4cba-bb7b-a5a1b2673dbf None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:16.047011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca65fb-d6f6-4cba-bb7b-a5a1b2673dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:16.047276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca65fb-d6f6-4cba-bb7b-a5a1b2673dbf None admin] PATH_INFO: `/v3/projects/6942c63862aa4a4996be8098c9e5b569` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:16.048479 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca65fb-d6f6-4cba-bb7b-a5a1b2673dbf None admin] RBAC: Authorizing `identity:get_project(project_id=6942c63862aa4a4996be8098c9e5b569)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:16.049906 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca65fb-d6f6-4cba-bb7b-a5a1b2673dbf None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:16.051573 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 567/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 07:01:15 2022] GET /identity/v3/projects/6942c63862aa4a4996be8098c9e5b569 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:16.058436 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981349d3-2935-4704-963a-08bc8b56a7e7 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:16.162761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981349d3-2935-4704-963a-08bc8b56a7e7 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:16.197959 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ead22a-40c2-4993-931e-887000cbecdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:16.198591 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ead22a-40c2-4993-931e-887000cbecdc None admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:16.198864 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ead22a-40c2-4993-931e-887000cbecdc None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:16.199131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ead22a-40c2-4993-931e-887000cbecdc None admin] PATH_INFO: `/v3/projects/6942c63862aa4a4996be8098c9e5b569/users/b1a27cd7cc0d412790057890164e9bb6/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:16.201167 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ead22a-40c2-4993-931e-887000cbecdc None admin] RBAC: Authorizing `identity:create_grant(project_id=6942c63862aa4a4996be8098c9e5b569, user_id=b1a27cd7cc0d412790057890164e9bb6, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:16.202595 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ead22a-40c2-4993-931e-887000cbecdc None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:16.222711 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 586/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 07:01:16 2022] PUT /identity/v3/projects/6942c63862aa4a4996be8098c9e5b569/users/b1a27cd7cc0d412790057890164e9bb6/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:01:18.088708 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df869cb-41ac-4dd4-bab7-668d2e92866d None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:18.089512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df869cb-41ac-4dd4-bab7-668d2e92866d None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:18.089512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df869cb-41ac-4dd4-bab7-668d2e92866d None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:18.091561 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 568/1154] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:01:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:01:18.097780 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bb2b8-f576-4cd4-8ada-a2d3db6fb111 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:18.098503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bb2b8-f576-4cd4-8ada-a2d3db6fb111 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:18.098648 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bb2b8-f576-4cd4-8ada-a2d3db6fb111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:18.125189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-981bb2b8-f576-4cd4-8ada-a2d3db6fb111 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:01:18.126389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-981bb2b8-f576-4cd4-8ada-a2d3db6fb111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:01:18.159445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:18.199874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 587/1155] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:01:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:01:18.468320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df869cb-41ac-4dd4-bab7-668d2e92866d None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:18.595115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df869cb-41ac-4dd4-bab7-668d2e92866d None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:18.633513 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b7dac-3db9-48a7-9a47-b692ecd3a0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:18.634123 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b7dac-3db9-48a7-9a47-b692ecd3a0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:18.634399 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b7dac-3db9-48a7-9a47-b692ecd3a0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:18.634667 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b7dac-3db9-48a7-9a47-b692ecd3a0e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:18.645777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b7dac-3db9-48a7-9a47-b692ecd3a0e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:01:18.650515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-067b7dac-3db9-48a7-9a47-b692ecd3a0e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:01:18.650515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:01:18.650515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:01:18.650515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:01:18.650515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:01:18.650515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:01:18.650515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:01:18.650515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:01:18.650515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:01:18.650515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:01:18.650515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:01:18.650515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:01:18.650515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:01:18.650515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:01:18.650515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:01:18.650515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:01:18.651508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:01:18.651508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:01:18.651508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:01:18.651508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:01:18.651508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:01:18.651508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:01:18.651508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:01:18.651508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:01:18.651508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:01:18.651508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:01:18.651508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:01:18.651508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:01:18.651508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:01:18.651508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:01:18.651508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:01:18.651508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:01:18.652374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:01:18.652374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:01:18.652374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:01:18.652374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:01:18.652374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:01:18.652374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:01:18.652374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:01:18.652374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:01:18.652374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:01:18.652374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:01:18.652374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:01:18.652374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:01:18.652374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 569/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:01:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:01:18.658114 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bb2b8-f576-4cd4-8ada-a2d3db6fb111 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:18.776156 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bb2b8-f576-4cd4-8ada-a2d3db6fb111 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:18.811892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc364954-18f4-4ea8-a9b9-4a54dce7c6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:18.812491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc364954-18f4-4ea8-a9b9-4a54dce7c6bc None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:18.812760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc364954-18f4-4ea8-a9b9-4a54dce7c6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:18.813026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc364954-18f4-4ea8-a9b9-4a54dce7c6bc None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:18.813681 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc364954-18f4-4ea8-a9b9-4a54dce7c6bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:01:18.813988 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc364954-18f4-4ea8-a9b9-4a54dce7c6bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:18.815351 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc364954-18f4-4ea8-a9b9-4a54dce7c6bc None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:18.824010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 588/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 07:01:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:01:18.833647 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b7dac-3db9-48a7-9a47-b692ecd3a0e0 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:18.901506 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:18.943535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b7dac-3db9-48a7-9a47-b692ecd3a0e0 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:18.993690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832902d-e950-45b6-9e0d-97cb96679bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:18.994315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832902d-e950-45b6-9e0d-97cb96679bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:18.994594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832902d-e950-45b6-9e0d-97cb96679bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:18.994866 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832902d-e950-45b6-9e0d-97cb96679bc7 None admin] PATH_INFO: `/v3/users/b1a27cd7cc0d412790057890164e9bb6` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:18.996140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832902d-e950-45b6-9e0d-97cb96679bc7 None admin] RBAC: Authorizing `identity:get_user(user_id=b1a27cd7cc0d412790057890164e9bb6)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:18.997603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832902d-e950-45b6-9e0d-97cb96679bc7 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:18.999209 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 570/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 07:01:18 2022] GET /identity/v3/users/b1a27cd7cc0d412790057890164e9bb6 => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:19.008124 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc364954-18f4-4ea8-a9b9-4a54dce7c6bc None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:19.042252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:19.115340 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc364954-18f4-4ea8-a9b9-4a54dce7c6bc None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:19.151692 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbcf10c-3b01-4bfe-a071-f6a4fd5d0243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:19.152294 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbcf10c-3b01-4bfe-a071-f6a4fd5d0243 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:19.152556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbcf10c-3b01-4bfe-a071-f6a4fd5d0243 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:19.152822 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbcf10c-3b01-4bfe-a071-f6a4fd5d0243 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:19.222597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbcf10c-3b01-4bfe-a071-f6a4fd5d0243 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:19.224063 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbcf10c-3b01-4bfe-a071-f6a4fd5d0243 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:19.261520 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:19.333987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 589/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:01:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:01:19.340467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832902d-e950-45b6-9e0d-97cb96679bc7 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:19.454727 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832902d-e950-45b6-9e0d-97cb96679bc7 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:19.483100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:19.496074 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2105e-d287-4716-bfb9-0ab693291873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:19.496709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2105e-d287-4716-bfb9-0ab693291873 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:19.496990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2105e-d287-4716-bfb9-0ab693291873 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:19.497262 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2105e-d287-4716-bfb9-0ab693291873 None admin] PATH_INFO: `/v3/projects/6942c63862aa4a4996be8098c9e5b569` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:19.498457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2105e-d287-4716-bfb9-0ab693291873 None admin] RBAC: Authorizing `identity:get_project(project_id=6942c63862aa4a4996be8098c9e5b569)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:19.499955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2105e-d287-4716-bfb9-0ab693291873 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:19.501604 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 571/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 07:01:19 2022] GET /identity/v3/projects/6942c63862aa4a4996be8098c9e5b569 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:19.508111 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbcf10c-3b01-4bfe-a071-f6a4fd5d0243 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:19.612845 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbcf10c-3b01-4bfe-a071-f6a4fd5d0243 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:19.642596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:19.649629 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26391fe-5123-461c-974f-44c7d04cd321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:19.650238 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26391fe-5123-461c-974f-44c7d04cd321 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:19.650519 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26391fe-5123-461c-974f-44c7d04cd321 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:19.650815 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26391fe-5123-461c-974f-44c7d04cd321 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:19.651570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26391fe-5123-461c-974f-44c7d04cd321 None admin] RBAC: Adding query filter params (role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.project.id=6942c63862aa4a4996be8098c9e5b569, user.id=b1a27cd7cc0d412790057890164e9bb6) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:01:19.651878 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26391fe-5123-461c-974f-44c7d04cd321 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:19.653325 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26391fe-5123-461c-974f-44c7d04cd321 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:19.661814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 590/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:01:19 2022] GET /identity/v3/role_assignments?user.id=b1a27cd7cc0d412790057890164e9bb6&scope.project.id=6942c63862aa4a4996be8098c9e5b569&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:21.562283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbb077-d781-4c97-a4d6-20dae4f32286 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:21.562876 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbb077-d781-4c97-a4d6-20dae4f32286 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:21.562876 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbb077-d781-4c97-a4d6-20dae4f32286 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:21.564280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 572/1162] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:01:21.570673 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e9a74-3e48-48d6-bc65-b75cde05046f None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:21.571100 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e9a74-3e48-48d6-bc65-b75cde05046f None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:21.571346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e9a74-3e48-48d6-bc65-b75cde05046f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:21.601550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-648e9a74-3e48-48d6-bc65-b75cde05046f None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:01:21.602837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-648e9a74-3e48-48d6-bc65-b75cde05046f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:01:21.673411 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 591/1163] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:01:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:01:21.935670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbb077-d781-4c97-a4d6-20dae4f32286 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:22.042725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbb077-d781-4c97-a4d6-20dae4f32286 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:22.078385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a25022-c948-4975-bdcc-c1c5878a90ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:22.078968 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a25022-c948-4975-bdcc-c1c5878a90ac None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:22.079240 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a25022-c948-4975-bdcc-c1c5878a90ac None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:22.079530 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a25022-c948-4975-bdcc-c1c5878a90ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:22.142774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:22.153071 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a25022-c948-4975-bdcc-c1c5878a90ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:22.154522 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a25022-c948-4975-bdcc-c1c5878a90ac None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:22.254760 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 573/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:01:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:01:22.261892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e9a74-3e48-48d6-bc65-b75cde05046f None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:22.378908 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e9a74-3e48-48d6-bc65-b75cde05046f None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:22.404395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:22.416817 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1bccae-95e6-4de1-9ac9-83c100b0fd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:22.417437 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bccae-95e6-4de1-9ac9-83c100b0fd8b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:22.417713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bccae-95e6-4de1-9ac9-83c100b0fd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:22.417983 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bccae-95e6-4de1-9ac9-83c100b0fd8b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:22.425107 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1bccae-95e6-4de1-9ac9-83c100b0fd8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:22.426547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1bccae-95e6-4de1-9ac9-83c100b0fd8b None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:22.438234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-aa1bccae-95e6-4de1-9ac9-83c100b0fd8b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 23 07:01:22.438234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:01:22.438234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 23 07:01:22.438234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 23 07:01:22.438234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:01:22.438234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:01:22.438234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:01:22.438234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:01:22.438234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 23 07:01:22.438234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:01:22.438234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:01:22.438234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:01:22.438234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:01:22.438234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:01:22.438234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:01:22.438234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:01:22.438234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:01:22.439389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:01:22.439389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:01:22.439389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:01:22.439389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:01:22.439389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:01:22.439389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:01:22.439389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:01:22.439389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:01:22.439389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:01:22.439389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:01:22.439389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:01:22.439389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:01:22.439389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:01:22.439389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:01:22.439389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:01:22.439389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:01:22.440461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:01:22.440461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:01:22.440461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:01:22.440461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:01:22.440461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:01:22.440461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:01:22.440461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:01:22.440461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:01:22.440461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:01:22.440461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:01:22.440461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 23 07:01:22.440461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:01:22.440461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 23 07:01:22.440461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:01:22.440461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 592/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 23 07:01:22 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:01:22.445854 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a25022-c948-4975-bdcc-c1c5878a90ac None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:22.547400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:22.554790 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a25022-c948-4975-bdcc-c1c5878a90ac None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:22.595980 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f75e5-8c3d-4767-8963-cf25253d0433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:22.596606 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f75e5-8c3d-4767-8963-cf25253d0433 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:22.596832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f75e5-8c3d-4767-8963-cf25253d0433 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:22.597099 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f75e5-8c3d-4767-8963-cf25253d0433 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:22.597729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f75e5-8c3d-4767-8963-cf25253d0433 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:01:22.598023 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f75e5-8c3d-4767-8963-cf25253d0433 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:22.599396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f75e5-8c3d-4767-8963-cf25253d0433 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:22.607558 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 574/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Aug 23 07:01:22 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:24.465791 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c01bb-f969-4339-bfc8-97328d8289dc None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:24.466640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c01bb-f969-4339-bfc8-97328d8289dc None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:24.466640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c01bb-f969-4339-bfc8-97328d8289dc None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:24.468838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 593/1167] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:01:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:01:24.476882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea086518-a6b8-4c79-b994-62e99051b191 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:24.477335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea086518-a6b8-4c79-b994-62e99051b191 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:24.477435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea086518-a6b8-4c79-b994-62e99051b191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:24.496321 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-ea086518-a6b8-4c79-b994-62e99051b191 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:01:24.497498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ea086518-a6b8-4c79-b994-62e99051b191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:01:24.569266 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 575/1168] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:01:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:01:24.837155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c01bb-f969-4339-bfc8-97328d8289dc None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:24.954732 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c01bb-f969-4339-bfc8-97328d8289dc None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:24.990449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c992f-8d03-4763-bc57-33f8f2003391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:24.991038 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c992f-8d03-4763-bc57-33f8f2003391 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:24.991309 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c992f-8d03-4763-bc57-33f8f2003391 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:24.991690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c992f-8d03-4763-bc57-33f8f2003391 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:24.992349 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c992f-8d03-4763-bc57-33f8f2003391 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:24.993721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c992f-8d03-4763-bc57-33f8f2003391 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:25.011707 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 594/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Tue Aug 23 07:01:24 2022] POST /identity/v3/domains => generated 267 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:01:26.960866 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8945f6c-19d8-41bc-983e-b0e5cb9cb90b None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:26.961407 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8945f6c-19d8-41bc-983e-b0e5cb9cb90b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:26.961407 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8945f6c-19d8-41bc-983e-b0e5cb9cb90b None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:26.962636 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 576/1170] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:01:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:01:26.968315 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eca774-6751-4d21-a3b1-85f6727a32ad None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:26.968745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eca774-6751-4d21-a3b1-85f6727a32ad None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:26.968834 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eca774-6751-4d21-a3b1-85f6727a32ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:26.988133 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-09eca774-6751-4d21-a3b1-85f6727a32ad None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:01:26.989325 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-09eca774-6751-4d21-a3b1-85f6727a32ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:01:27.046670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:27.092525 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 595/1171] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:01:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:01:27.363645 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8945f6c-19d8-41bc-983e-b0e5cb9cb90b None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:27.483790 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:27.501624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8945f6c-19d8-41bc-983e-b0e5cb9cb90b None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:27.540524 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c045f-394c-4620-8d29-0d9a4f1eb01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:27.541151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c045f-394c-4620-8d29-0d9a4f1eb01e None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:27.541428 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c045f-394c-4620-8d29-0d9a4f1eb01e None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:27.541701 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c045f-394c-4620-8d29-0d9a4f1eb01e None admin] PATH_INFO: `/v3/domains/d448b9b85db24828b2ecf90245e50984` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:27.552601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c045f-394c-4620-8d29-0d9a4f1eb01e None admin] RBAC: Authorizing `identity:get_domain(domain_id=d448b9b85db24828b2ecf90245e50984)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:27.554080 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c045f-394c-4620-8d29-0d9a4f1eb01e None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:27.555732 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 577/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Tue Aug 23 07:01:27 2022] GET /identity/v3/domains/d448b9b85db24828b2ecf90245e50984 => generated 267 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:27.562570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eca774-6751-4d21-a3b1-85f6727a32ad None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:27.676636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eca774-6751-4d21-a3b1-85f6727a32ad None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:27.712107 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734683f-6023-49c1-a36e-971c68112b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:27.712748 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734683f-6023-49c1-a36e-971c68112b76 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:27.713024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734683f-6023-49c1-a36e-971c68112b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:27.713294 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734683f-6023-49c1-a36e-971c68112b76 None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:27.713943 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734683f-6023-49c1-a36e-971c68112b76 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:27.715361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734683f-6023-49c1-a36e-971c68112b76 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:27.745761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 596/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 23 07:01:27 2022] POST /identity/v3/projects => generated 361 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:01:29.668543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74848f-ae19-4e9a-8d4c-2cc2b3538aa2 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:29.669005 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74848f-ae19-4e9a-8d4c-2cc2b3538aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:29.669098 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74848f-ae19-4e9a-8d4c-2cc2b3538aa2 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:29.671222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 578/1174] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:01:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:01:29.676652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91989d64-3975-4d4a-9482-752758eb720a None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:29.677072 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91989d64-3975-4d4a-9482-752758eb720a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:29.677230 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91989d64-3975-4d4a-9482-752758eb720a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:29.699735 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-91989d64-3975-4d4a-9482-752758eb720a None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:01:29.700933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-91989d64-3975-4d4a-9482-752758eb720a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:01:29.722561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:29.774449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 597/1175] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:01:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:01:30.045630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74848f-ae19-4e9a-8d4c-2cc2b3538aa2 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:30.171677 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74848f-ae19-4e9a-8d4c-2cc2b3538aa2 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:30.211892 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10521477-2217-445a-869c-17e3c6b571cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:30.212547 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10521477-2217-445a-869c-17e3c6b571cf None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:30.212822 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10521477-2217-445a-869c-17e3c6b571cf None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:30.213097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10521477-2217-445a-869c-17e3c6b571cf None admin] PATH_INFO: `/v3/domains/d448b9b85db24828b2ecf90245e50984` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:30.214313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10521477-2217-445a-869c-17e3c6b571cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=d448b9b85db24828b2ecf90245e50984)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:30.215823 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10521477-2217-445a-869c-17e3c6b571cf None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:30.217403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 579/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Tue Aug 23 07:01:30 2022] GET /identity/v3/domains/d448b9b85db24828b2ecf90245e50984 => generated 267 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:30.224275 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91989d64-3975-4d4a-9482-752758eb720a None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:30.342143 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91989d64-3975-4d4a-9482-752758eb720a None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:30.378723 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c67045-985f-4e51-9040-79df4ebcb404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:30.379327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c67045-985f-4e51-9040-79df4ebcb404 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:30.379640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c67045-985f-4e51-9040-79df4ebcb404 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:30.379916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c67045-985f-4e51-9040-79df4ebcb404 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:30.380569 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c67045-985f-4e51-9040-79df4ebcb404 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:30.382122 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c67045-985f-4e51-9040-79df4ebcb404 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:30.412393 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 598/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 23 07:01:30 2022] POST /identity/v3/users => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:01:32.379370 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2ecc4-4195-4518-ad0c-14b549ca80d2 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:32.379994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2ecc4-4195-4518-ad0c-14b549ca80d2 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:32.379994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2ecc4-4195-4518-ad0c-14b549ca80d2 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:32.381319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 580/1178] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:01:32.388334 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fea46f-5596-4941-9688-9573db9cc53a None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:32.389011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fea46f-5596-4941-9688-9573db9cc53a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:32.389329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fea46f-5596-4941-9688-9573db9cc53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:32.413459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-97fea46f-5596-4941-9688-9573db9cc53a None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:01:32.414664 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-97fea46f-5596-4941-9688-9573db9cc53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:01:32.422243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:32.486900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 599/1179] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:01:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:01:32.759676 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2ecc4-4195-4518-ad0c-14b549ca80d2 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:32.793289 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:32.896893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2ecc4-4195-4518-ad0c-14b549ca80d2 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:32.933092 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bdd08-ea7d-4dc8-8d99-5cc02943a5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:32.933647 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bdd08-ea7d-4dc8-8d99-5cc02943a5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:32.933929 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bdd08-ea7d-4dc8-8d99-5cc02943a5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:32.934189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bdd08-ea7d-4dc8-8d99-5cc02943a5d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:32.946014 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571bdd08-ea7d-4dc8-8d99-5cc02943a5d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:01:32.953730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-571bdd08-ea7d-4dc8-8d99-5cc02943a5d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:01:32.953730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:01:32.953730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:01:32.953730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:01:32.953730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:01:32.953730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:01:32.953730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:01:32.953730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:01:32.953730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:01:32.953730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:01:32.953730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:01:32.953730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:01:32.953730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:01:32.953730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:01:32.953730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:01:32.953730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:01:32.954802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:01:32.954802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:01:32.954802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:01:32.954802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:01:32.954802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:01:32.954802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:01:32.954802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:01:32.954802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:01:32.954802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:01:32.954802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:01:32.954802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:01:32.954802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:01:32.954802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:01:32.954802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:01:32.954802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:01:32.954802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:01:32.955828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:01:32.955828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:01:32.955828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:01:32.955828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:01:32.955828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:01:32.955828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:01:32.955828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:01:32.955828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:01:32.955828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:01:32.955828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:01:32.955828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:01:32.955828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:01:32.955828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 581/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:01:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:01:32.962803 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fea46f-5596-4941-9688-9573db9cc53a None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:33.073388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fea46f-5596-4941-9688-9573db9cc53a None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:33.114641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d2ea2-cc11-4dbc-80c4-6da8b3249680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:33.115241 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d2ea2-cc11-4dbc-80c4-6da8b3249680 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:33.115541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d2ea2-cc11-4dbc-80c4-6da8b3249680 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:33.115821 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d2ea2-cc11-4dbc-80c4-6da8b3249680 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:33.116570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d2ea2-cc11-4dbc-80c4-6da8b3249680 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:01:33.116866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d2ea2-cc11-4dbc-80c4-6da8b3249680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:33.118247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d2ea2-cc11-4dbc-80c4-6da8b3249680 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:33.126702 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 600/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 07:01:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:33.133764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bdd08-ea7d-4dc8-8d99-5cc02943a5d4 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:33.234124 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:33.260087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bdd08-ea7d-4dc8-8d99-5cc02943a5d4 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:33.307216 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d34cd-73bb-4fc0-884f-ed204a18bb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:33.307883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d34cd-73bb-4fc0-884f-ed204a18bb60 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:33.308163 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d34cd-73bb-4fc0-884f-ed204a18bb60 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:33.308433 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d34cd-73bb-4fc0-884f-ed204a18bb60 None admin] PATH_INFO: `/v3/users/dbbfce454d354a77914f98aab83e6860` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:33.323369 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d34cd-73bb-4fc0-884f-ed204a18bb60 None admin] RBAC: Authorizing `identity:get_user(user_id=dbbfce454d354a77914f98aab83e6860)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:33.324863 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d34cd-73bb-4fc0-884f-ed204a18bb60 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:33.326467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 582/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 07:01:33 2022] GET /identity/v3/users/dbbfce454d354a77914f98aab83e6860 => generated 311 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:33.332949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d2ea2-cc11-4dbc-80c4-6da8b3249680 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:33.337279 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:33.436037 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d2ea2-cc11-4dbc-80c4-6da8b3249680 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:33.471251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb5021-91e7-46b6-8712-91876f06dff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:33.471879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb5021-91e7-46b6-8712-91876f06dff5 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:33.472152 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb5021-91e7-46b6-8712-91876f06dff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:33.472413 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb5021-91e7-46b6-8712-91876f06dff5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:33.540679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb5021-91e7-46b6-8712-91876f06dff5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:33.542040 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb5021-91e7-46b6-8712-91876f06dff5 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:33.544621 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:33.645192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 601/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:01:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:01:33.651790 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d34cd-73bb-4fc0-884f-ed204a18bb60 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:33.756985 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d34cd-73bb-4fc0-884f-ed204a18bb60 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:33.797680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf0098-bfcc-4bb1-a70a-a3c838750c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:33.798221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf0098-bfcc-4bb1-a70a-a3c838750c0d None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:33.798494 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf0098-bfcc-4bb1-a70a-a3c838750c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:33.798764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf0098-bfcc-4bb1-a70a-a3c838750c0d None admin] PATH_INFO: `/v3/projects/0f6f4ea45ead478c8b7d32a86f56c136` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:33.800017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf0098-bfcc-4bb1-a70a-a3c838750c0d None admin] RBAC: Authorizing `identity:get_project(project_id=0f6f4ea45ead478c8b7d32a86f56c136)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:33.801460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf0098-bfcc-4bb1-a70a-a3c838750c0d None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:33.803086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 583/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 07:01:33 2022] GET /identity/v3/projects/0f6f4ea45ead478c8b7d32a86f56c136 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:33.810144 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb5021-91e7-46b6-8712-91876f06dff5 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:33.914819 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb5021-91e7-46b6-8712-91876f06dff5 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:33.917189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:33.949880 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf1972-7f61-4861-be70-5f51eebfc09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:33.950479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf1972-7f61-4861-be70-5f51eebfc09e None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:33.950773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf1972-7f61-4861-be70-5f51eebfc09e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:33.951019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf1972-7f61-4861-be70-5f51eebfc09e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:33.951794 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf1972-7f61-4861-be70-5f51eebfc09e None admin] RBAC: Adding query filter params (role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.project.id=0f6f4ea45ead478c8b7d32a86f56c136, user.id=dbbfce454d354a77914f98aab83e6860) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:01:33.952099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf1972-7f61-4861-be70-5f51eebfc09e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:33.953532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf1972-7f61-4861-be70-5f51eebfc09e None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:33.959667 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 602/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:01:33 2022] GET /identity/v3/role_assignments?user.id=dbbfce454d354a77914f98aab83e6860&scope.project.id=0f6f4ea45ead478c8b7d32a86f56c136&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:35.826221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf79c8e-364c-43b5-9579-8f668d6a81ff None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:35.826781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf79c8e-364c-43b5-9579-8f668d6a81ff None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:35.826781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf79c8e-364c-43b5-9579-8f668d6a81ff None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:35.828130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 584/1186] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:01:35.837132 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062a229-de8d-4137-8528-253b9f2742e7 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:35.837589 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062a229-de8d-4137-8528-253b9f2742e7 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:35.837681 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062a229-de8d-4137-8528-253b9f2742e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:35.867390 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-e062a229-de8d-4137-8528-253b9f2742e7 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:01:35.868619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e062a229-de8d-4137-8528-253b9f2742e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:01:35.941584 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 603/1187] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:01:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:01:36.208987 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf79c8e-364c-43b5-9579-8f668d6a81ff None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:36.213659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:36.323442 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf79c8e-364c-43b5-9579-8f668d6a81ff None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:36.363177 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661a2b6-e2df-4f8f-b2bc-bbef294f5eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:36.363815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661a2b6-e2df-4f8f-b2bc-bbef294f5eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:36.364087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661a2b6-e2df-4f8f-b2bc-bbef294f5eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:36.364358 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661a2b6-e2df-4f8f-b2bc-bbef294f5eb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:36.375060 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661a2b6-e2df-4f8f-b2bc-bbef294f5eb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:01:36.379707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-9661a2b6-e2df-4f8f-b2bc-bbef294f5eb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:01:36.379707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:01:36.379707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:01:36.379707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:01:36.379707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:01:36.379707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:01:36.379707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:01:36.379707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:01:36.379707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:01:36.379707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:01:36.379707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:01:36.379707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:01:36.379707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:01:36.379707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:01:36.379707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:01:36.379707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:01:36.380762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:01:36.380762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:01:36.380762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:01:36.380762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:01:36.380762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:01:36.380762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:01:36.380762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:01:36.380762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:01:36.380762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:01:36.380762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:01:36.380762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:01:36.380762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:01:36.380762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:01:36.380762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:01:36.380762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:01:36.380762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:01:36.382086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:01:36.382086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:01:36.382086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:01:36.382086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:01:36.382086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:01:36.382086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:01:36.382086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:01:36.382086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:01:36.382086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:01:36.382086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:01:36.382086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:01:36.382086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:01:36.382086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 585/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:01:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:01:36.388841 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062a229-de8d-4137-8528-253b9f2742e7 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:36.502571 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:36.507913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062a229-de8d-4137-8528-253b9f2742e7 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:36.543269 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232934eb-f59d-496a-b544-be82799d9b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:36.543885 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232934eb-f59d-496a-b544-be82799d9b13 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:36.544157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232934eb-f59d-496a-b544-be82799d9b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:36.544424 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232934eb-f59d-496a-b544-be82799d9b13 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:36.545080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232934eb-f59d-496a-b544-be82799d9b13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:01:36.545372 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232934eb-f59d-496a-b544-be82799d9b13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:36.546747 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232934eb-f59d-496a-b544-be82799d9b13 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:36.555057 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 604/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 07:01:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:36.561660 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661a2b6-e2df-4f8f-b2bc-bbef294f5eb7 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:36.636606 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:36.677966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661a2b6-e2df-4f8f-b2bc-bbef294f5eb7 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:36.719664 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e0315-b12c-4c10-923d-e2535c6affcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:36.720288 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e0315-b12c-4c10-923d-e2535c6affcb None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:36.720566 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e0315-b12c-4c10-923d-e2535c6affcb None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:36.720837 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e0315-b12c-4c10-923d-e2535c6affcb None admin] PATH_INFO: `/v3/users/dbbfce454d354a77914f98aab83e6860` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:36.722016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e0315-b12c-4c10-923d-e2535c6affcb None admin] RBAC: Authorizing `identity:get_user(user_id=dbbfce454d354a77914f98aab83e6860)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:36.723495 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e0315-b12c-4c10-923d-e2535c6affcb None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:36.725062 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 586/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 07:01:36 2022] GET /identity/v3/users/dbbfce454d354a77914f98aab83e6860 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:36.731307 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232934eb-f59d-496a-b544-be82799d9b13 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:36.840249 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232934eb-f59d-496a-b544-be82799d9b13 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:36.875349 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17faa79-d988-43a2-b10f-88b247ed7512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:36.876005 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17faa79-d988-43a2-b10f-88b247ed7512 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:36.876275 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17faa79-d988-43a2-b10f-88b247ed7512 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:36.876541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17faa79-d988-43a2-b10f-88b247ed7512 None admin] PATH_INFO: `/v3/projects/0f6f4ea45ead478c8b7d32a86f56c136` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:36.877764 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17faa79-d988-43a2-b10f-88b247ed7512 None admin] RBAC: Authorizing `identity:get_project(project_id=0f6f4ea45ead478c8b7d32a86f56c136)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:36.879199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17faa79-d988-43a2-b10f-88b247ed7512 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:36.880881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 605/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 07:01:36 2022] GET /identity/v3/projects/0f6f4ea45ead478c8b7d32a86f56c136 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:36.887653 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e0315-b12c-4c10-923d-e2535c6affcb None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:36.991840 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e0315-b12c-4c10-923d-e2535c6affcb None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:37.022442 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:37.032960 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac446c-43f1-492a-ab86-d6f2debea028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:37.033616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac446c-43f1-492a-ab86-d6f2debea028 None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:37.033890 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac446c-43f1-492a-ab86-d6f2debea028 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:37.034155 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac446c-43f1-492a-ab86-d6f2debea028 None admin] PATH_INFO: `/v3/projects/0f6f4ea45ead478c8b7d32a86f56c136/users/dbbfce454d354a77914f98aab83e6860/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:37.036144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac446c-43f1-492a-ab86-d6f2debea028 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f6f4ea45ead478c8b7d32a86f56c136, user_id=dbbfce454d354a77914f98aab83e6860, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:37.037589 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac446c-43f1-492a-ab86-d6f2debea028 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:37.052239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 587/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 07:01:36 2022] PUT /identity/v3/projects/0f6f4ea45ead478c8b7d32a86f56c136/users/dbbfce454d354a77914f98aab83e6860/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:01:38.942051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142d51d-f4fe-4abe-8078-36b859f537f7 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:38.942559 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142d51d-f4fe-4abe-8078-36b859f537f7 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:38.942559 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142d51d-f4fe-4abe-8078-36b859f537f7 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:38.944688 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 606/1193] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:01:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:01:38.953780 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c1fc8-55f7-4611-a8ad-0efe07ae1ce2 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:38.954211 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c1fc8-55f7-4611-a8ad-0efe07ae1ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:38.954343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c1fc8-55f7-4611-a8ad-0efe07ae1ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:38.981397 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-296c1fc8-55f7-4611-a8ad-0efe07ae1ce2 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:01:38.982586 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-296c1fc8-55f7-4611-a8ad-0efe07ae1ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:01:39.061306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 588/1194] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:01:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:01:39.330127 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142d51d-f4fe-4abe-8078-36b859f537f7 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:39.367691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:39.449019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142d51d-f4fe-4abe-8078-36b859f537f7 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:39.485245 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaeae42-26bb-43f6-abb6-f90ef8fa132b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:39.485870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaeae42-26bb-43f6-abb6-f90ef8fa132b None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:39.486146 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaeae42-26bb-43f6-abb6-f90ef8fa132b None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:39.486420 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaeae42-26bb-43f6-abb6-f90ef8fa132b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:39.498206 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaeae42-26bb-43f6-abb6-f90ef8fa132b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:01:39.502896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-7aaeae42-26bb-43f6-abb6-f90ef8fa132b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:01:39.502896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:01:39.502896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:01:39.502896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:01:39.502896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:01:39.502896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:01:39.502896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:01:39.502896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:01:39.502896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:01:39.502896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:01:39.502896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:01:39.502896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:01:39.502896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:01:39.502896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:01:39.502896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:01:39.502896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:01:39.504004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:01:39.504004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:01:39.504004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:01:39.504004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:01:39.504004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:01:39.504004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:01:39.504004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:01:39.504004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:01:39.504004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:01:39.504004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:01:39.504004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:01:39.504004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:01:39.504004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:01:39.504004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:01:39.504004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:01:39.504004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:01:39.505080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:01:39.505080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:01:39.505080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:01:39.505080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:01:39.505080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:01:39.505080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:01:39.505080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:01:39.505080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:01:39.505080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:01:39.505080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:01:39.505080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:01:39.505080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:01:39.505080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 607/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:01:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:01:39.512431 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c1fc8-55f7-4611-a8ad-0efe07ae1ce2 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:39.616258 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c1fc8-55f7-4611-a8ad-0efe07ae1ce2 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:39.640057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:39.653547 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52384a-ecf5-4b36-847c-684113907f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:39.654163 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52384a-ecf5-4b36-847c-684113907f28 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:39.654440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52384a-ecf5-4b36-847c-684113907f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:39.654709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52384a-ecf5-4b36-847c-684113907f28 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:39.655380 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52384a-ecf5-4b36-847c-684113907f28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:01:39.656027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52384a-ecf5-4b36-847c-684113907f28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:39.657119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52384a-ecf5-4b36-847c-684113907f28 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:39.666100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 589/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 07:01:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:39.672192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaeae42-26bb-43f6-abb6-f90ef8fa132b None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:39.773249 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:39.786959 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaeae42-26bb-43f6-abb6-f90ef8fa132b None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:39.821124 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2455f71-98d2-431f-ad8a-8f9ec2ff5145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:39.821742 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2455f71-98d2-431f-ad8a-8f9ec2ff5145 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:39.822021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2455f71-98d2-431f-ad8a-8f9ec2ff5145 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:39.822295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2455f71-98d2-431f-ad8a-8f9ec2ff5145 None admin] PATH_INFO: `/v3/users/dbbfce454d354a77914f98aab83e6860` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:39.823574 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2455f71-98d2-431f-ad8a-8f9ec2ff5145 None admin] RBAC: Authorizing `identity:get_user(user_id=dbbfce454d354a77914f98aab83e6860)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:39.825024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2455f71-98d2-431f-ad8a-8f9ec2ff5145 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:39.826627 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 608/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Aug 23 07:01:39 2022] GET /identity/v3/users/dbbfce454d354a77914f98aab83e6860 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:39.836429 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52384a-ecf5-4b36-847c-684113907f28 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:39.946604 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52384a-ecf5-4b36-847c-684113907f28 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:39.985638 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac1d20-2ba5-4bd4-8d37-4756717df2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:39.986262 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac1d20-2ba5-4bd4-8d37-4756717df2bd None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:39.986497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac1d20-2ba5-4bd4-8d37-4756717df2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:39.986767 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac1d20-2ba5-4bd4-8d37-4756717df2bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:40.039065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:40.065406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac1d20-2ba5-4bd4-8d37-4756717df2bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:40.067116 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac1d20-2ba5-4bd4-8d37-4756717df2bd None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:40.172479 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 590/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:01:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:01:40.181303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2455f71-98d2-431f-ad8a-8f9ec2ff5145 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:40.286445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2455f71-98d2-431f-ad8a-8f9ec2ff5145 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:40.324084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2078c-c958-443b-999b-651131d289d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:40.324703 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2078c-c958-443b-999b-651131d289d7 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:40.324981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2078c-c958-443b-999b-651131d289d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:40.325253 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2078c-c958-443b-999b-651131d289d7 None admin] PATH_INFO: `/v3/projects/0f6f4ea45ead478c8b7d32a86f56c136` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:40.326427 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2078c-c958-443b-999b-651131d289d7 None admin] RBAC: Authorizing `identity:get_project(project_id=0f6f4ea45ead478c8b7d32a86f56c136)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:40.328069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2078c-c958-443b-999b-651131d289d7 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:40.329849 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 609/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Aug 23 07:01:40 2022] GET /identity/v3/projects/0f6f4ea45ead478c8b7d32a86f56c136 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:40.338935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac1d20-2ba5-4bd4-8d37-4756717df2bd None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:01:40.432177 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:40.449366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac1d20-2ba5-4bd4-8d37-4756717df2bd None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:01:40.488870 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b2548-c599-49f7-a5fc-a94d6bc2c14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:01:40.489469 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b2548-c599-49f7-a5fc-a94d6bc2c14f None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:01:40.489745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b2548-c599-49f7-a5fc-a94d6bc2c14f None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:01:40.490016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b2548-c599-49f7-a5fc-a94d6bc2c14f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:01:40.490741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b2548-c599-49f7-a5fc-a94d6bc2c14f None admin] RBAC: Adding query filter params (role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.project.id=0f6f4ea45ead478c8b7d32a86f56c136, user.id=dbbfce454d354a77914f98aab83e6860) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:01:40.491039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b2548-c599-49f7-a5fc-a94d6bc2c14f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:01:40.492618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b2548-c599-49f7-a5fc-a94d6bc2c14f None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:01:40.499251 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 591/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:01:40 2022] GET /identity/v3/role_assignments?user.id=dbbfce454d354a77914f98aab83e6860&scope.project.id=0f6f4ea45ead478c8b7d32a86f56c136&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:01:50.319081 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=88 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:01:50.496124 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=26 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:35.874540 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d1c13-b2f6-468f-b9ac-09c685acf5d4 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:35.875144 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d1c13-b2f6-468f-b9ac-09c685acf5d4 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:35.875144 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d1c13-b2f6-468f-b9ac-09c685acf5d4 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:35.876646 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 610/1201] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:02:35.882397 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f9137-dfd6-4329-b0f1-7e9edab840f2 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:35.882881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f9137-dfd6-4329-b0f1-7e9edab840f2 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:35.882967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f9137-dfd6-4329-b0f1-7e9edab840f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:35.907147 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-b97f9137-dfd6-4329-b0f1-7e9edab840f2 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:02:35.908641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b97f9137-dfd6-4329-b0f1-7e9edab840f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:02:35.997551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 592/1202] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:02:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:02:36.284068 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d1c13-b2f6-468f-b9ac-09c685acf5d4 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:36.422588 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d1c13-b2f6-468f-b9ac-09c685acf5d4 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:36.470470 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f21097-3581-49ed-a505-2af39784d861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:36.471105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f21097-3581-49ed-a505-2af39784d861 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:36.471389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f21097-3581-49ed-a505-2af39784d861 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:36.471720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f21097-3581-49ed-a505-2af39784d861 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:36.483302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f21097-3581-49ed-a505-2af39784d861 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:36.487370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f21097-3581-49ed-a505-2af39784d861 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:36.508298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-b0f21097-3581-49ed-a505-2af39784d861 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:36.508298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:36.508298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:36.508298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:36.508298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:36.508298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:36.508298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:36.508298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:36.508298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:36.508298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:36.508298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:36.508298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:36.508298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:02:36.508298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:02:36.508298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:02:36.508298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:02:36.509863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:36.509863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:36.509863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:36.509863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:36.509863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:36.509863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:36.509863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:36.509863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:36.509863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:36.509863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:36.509863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:36.509863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:36.509863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:36.509863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:36.509863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:36.509863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:36.509863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:02:36.511285 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:02:36.511285 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:36.511285 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:02:36.511285 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 611/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:02:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:02:36.520670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f9137-dfd6-4329-b0f1-7e9edab840f2 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:36.675599 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f9137-dfd6-4329-b0f1-7e9edab840f2 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:36.724803 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:36.732241 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977297d-51c0-4515-9647-f9c1e24fb9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:36.732837 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977297d-51c0-4515-9647-f9c1e24fb9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:36.733116 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977297d-51c0-4515-9647-f9c1e24fb9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:36.733386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977297d-51c0-4515-9647-f9c1e24fb9d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:36.734038 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977297d-51c0-4515-9647-f9c1e24fb9d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:36.734338 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977297d-51c0-4515-9647-f9c1e24fb9d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:36.735764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977297d-51c0-4515-9647-f9c1e24fb9d9 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:36.752794 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 593/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:36.759097 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f21097-3581-49ed-a505-2af39784d861 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:36.825965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:36.894789 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f21097-3581-49ed-a505-2af39784d861 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:36.955228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d90752-4a47-40f8-9c2f-6cbd42abc981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:36.956236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d90752-4a47-40f8-9c2f-6cbd42abc981 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:36.956690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d90752-4a47-40f8-9c2f-6cbd42abc981 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:36.956996 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d90752-4a47-40f8-9c2f-6cbd42abc981 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:36.957826 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d90752-4a47-40f8-9c2f-6cbd42abc981 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:36.959731 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d90752-4a47-40f8-9c2f-6cbd42abc981 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:36.986130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 612/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Aug 23 07:02:36 2022] POST /identity/v3/users => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:02:39.203329 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c527dcb-771c-4e4f-ad63-1ee7695ebb66 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:39.203926 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c527dcb-771c-4e4f-ad63-1ee7695ebb66 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:39.203926 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c527dcb-771c-4e4f-ad63-1ee7695ebb66 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:39.205721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 594/1206] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:02:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:02:39.212916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3511d-d9d0-4ab0-bcec-71d8a1365f75 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:39.213347 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3511d-d9d0-4ab0-bcec-71d8a1365f75 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:39.213604 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3511d-d9d0-4ab0-bcec-71d8a1365f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:39.233182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-bfc3511d-d9d0-4ab0-bcec-71d8a1365f75 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:02:39.234547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-bfc3511d-d9d0-4ab0-bcec-71d8a1365f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:02:39.312660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 613/1207] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:02:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:02:39.578745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c527dcb-771c-4e4f-ad63-1ee7695ebb66 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:39.705646 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c527dcb-771c-4e4f-ad63-1ee7695ebb66 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:39.743021 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f0186-0c4e-4482-8c83-44680f7bbfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:39.743766 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f0186-0c4e-4482-8c83-44680f7bbfa9 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:39.744045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f0186-0c4e-4482-8c83-44680f7bbfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:39.744382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f0186-0c4e-4482-8c83-44680f7bbfa9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:39.756335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f0186-0c4e-4482-8c83-44680f7bbfa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:02:39.761712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-471f0186-0c4e-4482-8c83-44680f7bbfa9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:02:39.761712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:39.761712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:39.761712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:39.761712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:39.761712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:39.761712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:39.761712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:39.761712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:39.761712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:39.761712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:39.761712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:39.761712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:02:39.761712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:02:39.761712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:02:39.761712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:02:39.763567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:02:39.763567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:02:39.763567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:02:39.763567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:02:39.763567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:39.763567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:39.763567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:39.763567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:39.763567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:39.763567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:39.763567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:39.763567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:39.763567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:39.763567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:39.763567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:39.763567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:39.764658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:39.764658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:39.764658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:39.764658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:39.764658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:02:39.764658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:02:39.764658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:02:39.764658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:02:39.764658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:02:39.764658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:02:39.764658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:02:39.764658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:02:39.764658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 595/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 07:02:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:02:39.768947 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3511d-d9d0-4ab0-bcec-71d8a1365f75 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:39.814790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:39.894258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3511d-d9d0-4ab0-bcec-71d8a1365f75 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:39.933427 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a2c53-f12e-4229-864a-f9e7d20cafd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:39.934156 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a2c53-f12e-4229-864a-f9e7d20cafd4 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:39.934414 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a2c53-f12e-4229-864a-f9e7d20cafd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:39.934707 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a2c53-f12e-4229-864a-f9e7d20cafd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:39.935509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a2c53-f12e-4229-864a-f9e7d20cafd4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:39.935806 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a2c53-f12e-4229-864a-f9e7d20cafd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:39.937390 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a2c53-f12e-4229-864a-f9e7d20cafd4 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:39.952906 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 614/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 07:02:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:02:39.954771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f0186-0c4e-4482-8c83-44680f7bbfa9 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:40.016643 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:40.080868 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f0186-0c4e-4482-8c83-44680f7bbfa9 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:40.121399 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeeda00-5547-477b-b75d-24a5c82e5a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:40.122029 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeeda00-5547-477b-b75d-24a5c82e5a61 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:40.122304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeeda00-5547-477b-b75d-24a5c82e5a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:40.122572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeeda00-5547-477b-b75d-24a5c82e5a61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:40.134231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeeda00-5547-477b-b75d-24a5c82e5a61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:40.136013 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeeda00-5547-477b-b75d-24a5c82e5a61 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:40.152222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-faeeda00-5547-477b-b75d-24a5c82e5a61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:40.152222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:40.152222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:40.152222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:40.152222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:40.152222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:40.152222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:40.152222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:40.152222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:40.152222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:40.152222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:40.152222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:40.152222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:02:40.152222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:02:40.152222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:02:40.152222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:02:40.153463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:40.153463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:40.153463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:40.153463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:40.153463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:40.153463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:40.153463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:40.153463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:40.153463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:40.153463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:40.153463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:40.153463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:40.153463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:40.153463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:40.153463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:40.153463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:40.153463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:02:40.154434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:02:40.154434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:40.154434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:02:40.154434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 596/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:02:40.162052 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a2c53-f12e-4229-864a-f9e7d20cafd4 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:40.280832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:40.284233 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a2c53-f12e-4229-864a-f9e7d20cafd4 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:40.328198 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce977b3-f91a-4262-94b9-3f7932bb6744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:40.329215 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce977b3-f91a-4262-94b9-3f7932bb6744 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:40.329711 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce977b3-f91a-4262-94b9-3f7932bb6744 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:40.330206 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce977b3-f91a-4262-94b9-3f7932bb6744 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:40.331400 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce977b3-f91a-4262-94b9-3f7932bb6744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:40.332164 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce977b3-f91a-4262-94b9-3f7932bb6744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:40.334249 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce977b3-f91a-4262-94b9-3f7932bb6744 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:40.350211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 615/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:02:40.356182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeeda00-5547-477b-b75d-24a5c82e5a61 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:40.463511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeeda00-5547-477b-b75d-24a5c82e5a61 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:40.465140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:40.500647 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad7142-fe71-4b08-acd7-699f0c36eb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:40.501287 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad7142-fe71-4b08-acd7-699f0c36eb43 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:40.501570 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad7142-fe71-4b08-acd7-699f0c36eb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:40.501845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad7142-fe71-4b08-acd7-699f0c36eb43 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:40.511466 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad7142-fe71-4b08-acd7-699f0c36eb43 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:40.512912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad7142-fe71-4b08-acd7-699f0c36eb43 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:40.523755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-b1ad7142-fe71-4b08-acd7-699f0c36eb43 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:02:40.523755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:40.523755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:40.523755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:40.523755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:40.523755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:40.523755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:40.523755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:40.523755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:40.523755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:40.523755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:40.523755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:40.523755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:02:40.523755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:02:40.523755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:02:40.523755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:02:40.529996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:40.529996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:40.529996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:02:40.529996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:02:40.529996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:02:40.529996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:02:40.529996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:40.529996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:40.529996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:40.529996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:40.529996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:40.529996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:40.529996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:40.529996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:40.529996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:40.529996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:40.529996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:40.529996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:40.531133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:40.531133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:40.531133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:02:40.531133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:02:40.531133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:02:40.531133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:02:40.531133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:02:40.531133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:02:40.531133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:02:40.531133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:02:40.531133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 597/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:02:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:02:40.531801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce977b3-f91a-4262-94b9-3f7932bb6744 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:40.643398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce977b3-f91a-4262-94b9-3f7932bb6744 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:40.682074 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b923ca7-eeee-4520-af6f-995734ae2e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:40.685459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b923ca7-eeee-4520-af6f-995734ae2e8c None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:40.685802 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b923ca7-eeee-4520-af6f-995734ae2e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:40.686088 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b923ca7-eeee-4520-af6f-995734ae2e8c None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:40.686953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b923ca7-eeee-4520-af6f-995734ae2e8c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:40.687289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b923ca7-eeee-4520-af6f-995734ae2e8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:40.688899 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b923ca7-eeee-4520-af6f-995734ae2e8c None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:40.711602 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 616/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Aug 23 07:02:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:40.716197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad7142-fe71-4b08-acd7-699f0c36eb43 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:40.849027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad7142-fe71-4b08-acd7-699f0c36eb43 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:40.892538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b202dd-e821-4b82-b510-743da257e45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:40.893461 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b202dd-e821-4b82-b510-743da257e45a None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:40.894180 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b202dd-e821-4b82-b510-743da257e45a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:40.894523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b202dd-e821-4b82-b510-743da257e45a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:40.934279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:40.986643 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b202dd-e821-4b82-b510-743da257e45a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:40.988153 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b202dd-e821-4b82-b510-743da257e45a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:41.103050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 598/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:02:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:02:41.109313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b923ca7-eeee-4520-af6f-995734ae2e8c None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:41.129986 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:41.220713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b923ca7-eeee-4520-af6f-995734ae2e8c None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:41.258003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbae0a8-c232-4381-ad5e-4966ac7f3b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:41.258581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbae0a8-c232-4381-ad5e-4966ac7f3b22 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:41.258907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbae0a8-c232-4381-ad5e-4966ac7f3b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:41.259134 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbae0a8-c232-4381-ad5e-4966ac7f3b22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:41.270186 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbae0a8-c232-4381-ad5e-4966ac7f3b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:41.271678 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbae0a8-c232-4381-ad5e-4966ac7f3b22 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:41.287268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-7cbae0a8-c232-4381-ad5e-4966ac7f3b22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:41.287268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:41.287268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:41.287268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:41.287268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:41.287268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:41.287268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:41.287268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:41.287268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:41.287268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:41.287268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:41.287268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:41.287268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:02:41.287268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:02:41.287268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:02:41.287268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:02:41.288916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:41.288916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:41.288916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:41.288916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:41.288916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:41.288916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:41.288916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:41.288916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:41.288916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:41.288916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:41.288916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:41.288916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:41.288916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:41.288916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:41.288916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:41.288916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:41.288916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:02:41.290008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:02:41.290008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:41.290008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:02:41.290008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 617/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:02:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:02:41.294333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b202dd-e821-4b82-b510-743da257e45a None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:41.365209 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:41.411033 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b202dd-e821-4b82-b510-743da257e45a None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:41.455289 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da3a36-a824-4bcb-ad91-d09e9a2ef5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:41.456519 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da3a36-a824-4bcb-ad91-d09e9a2ef5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:41.457128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da3a36-a824-4bcb-ad91-d09e9a2ef5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:41.457654 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da3a36-a824-4bcb-ad91-d09e9a2ef5d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:41.458685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da3a36-a824-4bcb-ad91-d09e9a2ef5d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:41.459385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da3a36-a824-4bcb-ad91-d09e9a2ef5d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:41.461313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da3a36-a824-4bcb-ad91-d09e9a2ef5d3 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:41.483385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 599/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:02:41.485122 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbae0a8-c232-4381-ad5e-4966ac7f3b22 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:41.562635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:41.599608 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbae0a8-c232-4381-ad5e-4966ac7f3b22 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:41.637198 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23837b46-b20c-4d02-bc86-b80a15010b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:41.637826 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23837b46-b20c-4d02-bc86-b80a15010b3e None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:41.638109 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23837b46-b20c-4d02-bc86-b80a15010b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:41.638377 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23837b46-b20c-4d02-bc86-b80a15010b3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:41.646304 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23837b46-b20c-4d02-bc86-b80a15010b3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:41.647810 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23837b46-b20c-4d02-bc86-b80a15010b3e None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:41.660572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-23837b46-b20c-4d02-bc86-b80a15010b3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:02:41.660572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:41.660572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:41.660572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:41.660572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:41.660572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:41.660572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:41.660572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:41.660572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:41.660572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:41.660572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:41.660572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:41.660572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:02:41.660572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:02:41.660572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:02:41.661494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:02:41.661494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:41.661494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:41.661494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:41.661494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:41.661494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:41.661494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:41.661494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:41.661494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:41.661494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:41.661494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:41.661494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:41.661494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:41.661494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:41.661494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:41.661494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:41.661494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:41.662445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:02:41.662445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:02:41.662445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:02:41.662445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:02:41.662445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:02:41.662445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:02:41.662445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:02:41.662445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:02:41.662445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 618/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:02:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:02:41.667665 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da3a36-a824-4bcb-ad91-d09e9a2ef5d3 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:41.786562 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da3a36-a824-4bcb-ad91-d09e9a2ef5d3 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:41.813176 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:41.826257 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbacbdcd-bd10-4e8c-8ecf-321a37e8d469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:41.827305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacbdcd-bd10-4e8c-8ecf-321a37e8d469 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:41.827761 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacbdcd-bd10-4e8c-8ecf-321a37e8d469 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:41.828036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacbdcd-bd10-4e8c-8ecf-321a37e8d469 None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:41.828941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbacbdcd-bd10-4e8c-8ecf-321a37e8d469 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:41.829264 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbacbdcd-bd10-4e8c-8ecf-321a37e8d469 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:41.831150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbacbdcd-bd10-4e8c-8ecf-321a37e8d469 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:41.851222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 600/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:02:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:02:41.852476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23837b46-b20c-4d02-bc86-b80a15010b3e None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:41.975237 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23837b46-b20c-4d02-bc86-b80a15010b3e None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:42.006675 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:42.014601 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94043b2a-1c5b-4feb-9853-a35953a1cdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:42.015241 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94043b2a-1c5b-4feb-9853-a35953a1cdb6 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:42.015630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94043b2a-1c5b-4feb-9853-a35953a1cdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:42.015921 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94043b2a-1c5b-4feb-9853-a35953a1cdb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:42.016749 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94043b2a-1c5b-4feb-9853-a35953a1cdb6 None admin] RBAC: Adding query filter params (role.id=2d62b8b8395745c7983c6c8d100d985b, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=a9b0f54c988e43bda474f978a69fa4fe) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:42.017041 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94043b2a-1c5b-4feb-9853-a35953a1cdb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:42.019040 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94043b2a-1c5b-4feb-9853-a35953a1cdb6 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:42.034474 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 619/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:02:41 2022] GET /identity/v3/role_assignments?user.id=a9b0f54c988e43bda474f978a69fa4fe&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2d62b8b8395745c7983c6c8d100d985b => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:43.948776 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616db76-8278-4dca-9565-18d83d94c4d0 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:43.949304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616db76-8278-4dca-9565-18d83d94c4d0 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:43.949304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616db76-8278-4dca-9565-18d83d94c4d0 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:43.950903 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 601/1220] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:02:43.956801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5e126-f5b6-40f3-9fff-c2ace1c842a9 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:43.957358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5e126-f5b6-40f3-9fff-c2ace1c842a9 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:43.957358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5e126-f5b6-40f3-9fff-c2ace1c842a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:43.978738 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-9ed5e126-f5b6-40f3-9fff-c2ace1c842a9 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:02:43.980314 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9ed5e126-f5b6-40f3-9fff-c2ace1c842a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:02:44.067247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 620/1221] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:02:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:02:44.353064 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616db76-8278-4dca-9565-18d83d94c4d0 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:44.471743 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616db76-8278-4dca-9565-18d83d94c4d0 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:44.511005 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f5e10-f274-4a62-9416-d959c15b3fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:44.511672 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f5e10-f274-4a62-9416-d959c15b3fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:44.511954 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f5e10-f274-4a62-9416-d959c15b3fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:44.512228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f5e10-f274-4a62-9416-d959c15b3fb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:44.523277 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f5e10-f274-4a62-9416-d959c15b3fb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:02:44.528028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-a26f5e10-f274-4a62-9416-d959c15b3fb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:02:44.528028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:44.528028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:44.528028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:44.528028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:44.528028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:44.528028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:44.528028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:44.528028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:44.528028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:44.528028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:44.528028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:44.528028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:02:44.528028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:02:44.528028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:02:44.528028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:02:44.529121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:02:44.529121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:02:44.529121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:02:44.529121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:02:44.529121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:44.529121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:44.529121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:44.529121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:44.529121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:44.529121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:44.529121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:44.529121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:44.529121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:44.529121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:44.529121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:44.529121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:44.530039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:44.530039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:44.530039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:44.530039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:44.530039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:02:44.530039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:02:44.530039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:02:44.530039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:02:44.530039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:02:44.530039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:02:44.530039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:02:44.530039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:02:44.530039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 602/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 07:02:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:02:44.535243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5e126-f5b6-40f3-9fff-c2ace1c842a9 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:44.641434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5e126-f5b6-40f3-9fff-c2ace1c842a9 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:44.667246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:44.678697 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563851e2-0924-49ac-b272-2f166c0376b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:44.679310 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563851e2-0924-49ac-b272-2f166c0376b5 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:44.679613 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563851e2-0924-49ac-b272-2f166c0376b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:44.679883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563851e2-0924-49ac-b272-2f166c0376b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:44.680547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563851e2-0924-49ac-b272-2f166c0376b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:44.680844 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563851e2-0924-49ac-b272-2f166c0376b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:44.682227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563851e2-0924-49ac-b272-2f166c0376b5 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:44.693457 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 621/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 07:02:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:44.699250 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f5e10-f274-4a62-9416-d959c15b3fb4 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:44.747144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:44.813993 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f5e10-f274-4a62-9416-d959c15b3fb4 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:44.852601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb32fc-cd73-4f68-8bec-21cf3263a028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:44.853232 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb32fc-cd73-4f68-8bec-21cf3263a028 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:44.853545 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb32fc-cd73-4f68-8bec-21cf3263a028 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:44.853870 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb32fc-cd73-4f68-8bec-21cf3263a028 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:44.866797 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb32fc-cd73-4f68-8bec-21cf3263a028 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:44.868290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb32fc-cd73-4f68-8bec-21cf3263a028 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:44.884089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-d9bb32fc-cd73-4f68-8bec-21cf3263a028 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:44.884089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:44.884089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:44.884089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:44.884089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:44.884089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:44.884089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:44.884089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:44.884089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:44.884089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:44.884089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:44.884089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:44.884089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:02:44.884089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:02:44.884089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:02:44.884089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:02:44.885239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:44.885239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:44.885239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:44.885239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:44.885239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:44.885239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:44.885239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:44.885239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:44.885239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:44.885239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:44.885239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:44.885239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:44.885239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:44.885239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:44.885239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:44.885239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:44.885239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:02:44.886353 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:02:44.886353 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:44.886353 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:02:44.886353 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 603/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:02:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:02:44.892156 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563851e2-0924-49ac-b272-2f166c0376b5 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:45.008918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563851e2-0924-49ac-b272-2f166c0376b5 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:45.059464 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f5f65-cb72-457f-819a-f93d07a8b35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:45.060823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f5f65-cb72-457f-819a-f93d07a8b35d None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:45.061327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f5f65-cb72-457f-819a-f93d07a8b35d None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:45.061611 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f5f65-cb72-457f-819a-f93d07a8b35d None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:45.062272 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f5f65-cb72-457f-819a-f93d07a8b35d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:45.062573 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f5f65-cb72-457f-819a-f93d07a8b35d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:45.064119 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f5f65-cb72-457f-819a-f93d07a8b35d None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:45.079431 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 622/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:45.085570 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb32fc-cd73-4f68-8bec-21cf3263a028 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:45.186339 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:45.194033 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb32fc-cd73-4f68-8bec-21cf3263a028 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:45.229005 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06c6c2-c838-4c41-afed-fac95a0cd46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:45.229641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06c6c2-c838-4c41-afed-fac95a0cd46f None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:45.229923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06c6c2-c838-4c41-afed-fac95a0cd46f None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:45.230201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06c6c2-c838-4c41-afed-fac95a0cd46f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:45.237859 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd06c6c2-c838-4c41-afed-fac95a0cd46f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:45.239293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd06c6c2-c838-4c41-afed-fac95a0cd46f None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:45.253168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-fd06c6c2-c838-4c41-afed-fac95a0cd46f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:02:45.253168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:45.253168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:45.253168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:45.253168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:45.253168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:45.253168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:45.253168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:45.253168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:45.253168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:45.253168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:45.253168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:45.253168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:02:45.253168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:02:45.253168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:02:45.253168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:02:45.254406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:45.254406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:45.254406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:02:45.254406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:02:45.254406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:02:45.254406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:02:45.254406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:45.254406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:45.254406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:45.254406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:45.254406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:45.254406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:45.254406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:45.254406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:45.254406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:45.254406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:45.254406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:45.254406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:45.255382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:45.255382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:45.255382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:02:45.255382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:02:45.255382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:02:45.255382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:02:45.255382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:02:45.255382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:02:45.255382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:02:45.255382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:02:45.255382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 604/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:02:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:02:45.260497 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f5f65-cb72-457f-819a-f93d07a8b35d None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:45.311719 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:45.375701 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f5f65-cb72-457f-819a-f93d07a8b35d None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:45.419625 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb6bcb-8a6b-49d2-94eb-d7f0409cef44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:45.420229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb6bcb-8a6b-49d2-94eb-d7f0409cef44 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:45.420647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb6bcb-8a6b-49d2-94eb-d7f0409cef44 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:45.420925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb6bcb-8a6b-49d2-94eb-d7f0409cef44 None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:45.421662 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb6bcb-8a6b-49d2-94eb-d7f0409cef44 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:45.421969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb6bcb-8a6b-49d2-94eb-d7f0409cef44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:45.423427 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb6bcb-8a6b-49d2-94eb-d7f0409cef44 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:45.443871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 623/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Aug 23 07:02:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:45.451152 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06c6c2-c838-4c41-afed-fac95a0cd46f None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:45.570009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06c6c2-c838-4c41-afed-fac95a0cd46f None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:45.611564 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40387a05-810a-4a8f-8872-745b6cc4d91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:45.612332 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40387a05-810a-4a8f-8872-745b6cc4d91f None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:45.612675 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40387a05-810a-4a8f-8872-745b6cc4d91f None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:45.612951 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40387a05-810a-4a8f-8872-745b6cc4d91f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:45.625346 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40387a05-810a-4a8f-8872-745b6cc4d91f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:45.626936 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40387a05-810a-4a8f-8872-745b6cc4d91f None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:45.642393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-40387a05-810a-4a8f-8872-745b6cc4d91f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:45.642393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:45.642393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:45.642393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:45.642393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:45.642393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:45.642393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:45.642393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:45.642393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:45.642393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:45.642393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:45.642393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:45.642393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:02:45.642393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:02:45.642393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:02:45.642393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:02:45.649086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:45.649086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:45.649086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:45.649086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:45.649086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:45.649086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:45.649086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:45.649086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:45.649086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:45.649086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:45.649086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:45.649086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:45.649086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:45.649086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:45.649086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:45.649086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:45.649086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:02:45.650173 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:02:45.650173 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:45.650173 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:02:45.650173 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 605/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:02:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:02:45.650414 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb6bcb-8a6b-49d2-94eb-d7f0409cef44 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:45.763111 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb6bcb-8a6b-49d2-94eb-d7f0409cef44 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:45.765547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:45.800749 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c70ba-769c-4d7e-8a6a-f4362cf06fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:45.801409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c70ba-769c-4d7e-8a6a-f4362cf06fdf None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:45.801692 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c70ba-769c-4d7e-8a6a-f4362cf06fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:45.801964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c70ba-769c-4d7e-8a6a-f4362cf06fdf None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:45.802619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c70ba-769c-4d7e-8a6a-f4362cf06fdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:45.802926 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c70ba-769c-4d7e-8a6a-f4362cf06fdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:45.804345 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c70ba-769c-4d7e-8a6a-f4362cf06fdf None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:45.817894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 624/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:45.823869 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40387a05-810a-4a8f-8872-745b6cc4d91f None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:45.834519 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:45.946228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40387a05-810a-4a8f-8872-745b6cc4d91f None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:45.985818 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e66361-7485-426f-b2ea-49622a62f6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:45.986536 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e66361-7485-426f-b2ea-49622a62f6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:45.986823 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e66361-7485-426f-b2ea-49622a62f6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:45.987091 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e66361-7485-426f-b2ea-49622a62f6b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:46.001128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e66361-7485-426f-b2ea-49622a62f6b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:46.002788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e66361-7485-426f-b2ea-49622a62f6b1 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:46.015345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-81e66361-7485-426f-b2ea-49622a62f6b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:02:46.015345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:46.015345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:46.015345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:46.015345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:46.015345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:46.015345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:46.015345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:46.015345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:46.015345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:46.015345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:46.015345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:46.015345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:02:46.015345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:02:46.015345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:02:46.016336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:02:46.016336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:46.016336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:46.016336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:46.016336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:46.016336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:46.016336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:46.016336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:46.016336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:46.016336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:46.016336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:46.016336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:46.016336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:46.016336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:46.016336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:46.016336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:46.016336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:46.022889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:02:46.022889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:02:46.022889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:02:46.022889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:02:46.022889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:02:46.022889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:02:46.022889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:02:46.022889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:02:46.022889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 606/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:02:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:02:46.023538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c70ba-769c-4d7e-8a6a-f4362cf06fdf None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:46.137632 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c70ba-769c-4d7e-8a6a-f4362cf06fdf None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:46.174264 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a952ad-13e6-4b47-a0be-6cf5d302fbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:46.174949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a952ad-13e6-4b47-a0be-6cf5d302fbf2 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:46.175213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a952ad-13e6-4b47-a0be-6cf5d302fbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:46.175512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a952ad-13e6-4b47-a0be-6cf5d302fbf2 None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:46.176200 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a952ad-13e6-4b47-a0be-6cf5d302fbf2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:46.176485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a952ad-13e6-4b47-a0be-6cf5d302fbf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:46.177910 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a952ad-13e6-4b47-a0be-6cf5d302fbf2 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:46.190639 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 625/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:02:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:46.196979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e66361-7485-426f-b2ea-49622a62f6b1 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:46.267056 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:46.304785 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e66361-7485-426f-b2ea-49622a62f6b1 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:46.343319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d1c59-4112-4ea2-b03a-3bc9c3fd87e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:46.344137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d1c59-4112-4ea2-b03a-3bc9c3fd87e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:46.344445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d1c59-4112-4ea2-b03a-3bc9c3fd87e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:46.344802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d1c59-4112-4ea2-b03a-3bc9c3fd87e3 None admin] PATH_INFO: `/v3/projects/351a6777525c4207826d7df0d3768e99/users/a9b0f54c988e43bda474f978a69fa4fe/roles/2d62b8b8395745c7983c6c8d100d985b` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:46.363716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d1c59-4112-4ea2-b03a-3bc9c3fd87e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=351a6777525c4207826d7df0d3768e99, user_id=a9b0f54c988e43bda474f978a69fa4fe, role_id=2d62b8b8395745c7983c6c8d100d985b)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:46.365491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d1c59-4112-4ea2-b03a-3bc9c3fd87e3 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:46.381889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 607/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 07:02:46 2022] PUT /identity/v3/projects/351a6777525c4207826d7df0d3768e99/users/a9b0f54c988e43bda474f978a69fa4fe/roles/2d62b8b8395745c7983c6c8d100d985b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:02:48.416300 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f833d-9f9f-4345-8129-d152c79c00dc None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:48.416883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f833d-9f9f-4345-8129-d152c79c00dc None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:48.416883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f833d-9f9f-4345-8129-d152c79c00dc None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:48.418180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 626/1233] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:02:48.423702 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2ad94-d656-45f1-a85e-885aa5b7177a None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:48.424197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2ad94-d656-45f1-a85e-885aa5b7177a None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:48.425145 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2ad94-d656-45f1-a85e-885aa5b7177a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:48.445233 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-83e2ad94-d656-45f1-a85e-885aa5b7177a None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:02:48.446445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-83e2ad94-d656-45f1-a85e-885aa5b7177a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:02:48.519675 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 608/1234] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:02:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:02:48.787118 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f833d-9f9f-4345-8129-d152c79c00dc None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:48.812932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:48.896641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f833d-9f9f-4345-8129-d152c79c00dc None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:48.934825 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0bd40-b16e-4203-a1bb-5fe7f371e149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:48.935457 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0bd40-b16e-4203-a1bb-5fe7f371e149 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:48.935733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0bd40-b16e-4203-a1bb-5fe7f371e149 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:48.936004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0bd40-b16e-4203-a1bb-5fe7f371e149 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:48.947102 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0bd40-b16e-4203-a1bb-5fe7f371e149 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:02:48.952181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-02c0bd40-b16e-4203-a1bb-5fe7f371e149 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:02:48.952181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:48.952181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:48.952181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:48.952181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:48.952181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:48.952181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:48.952181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:48.952181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:48.952181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:48.952181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:48.952181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:48.952181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:02:48.952181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:02:48.952181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:02:48.952181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:02:48.953293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:02:48.953293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:02:48.953293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:02:48.953293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:02:48.953293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:48.953293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:48.953293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:48.953293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:48.953293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:48.953293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:48.953293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:48.953293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:48.953293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:48.953293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:48.953293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:48.953293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:48.954141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:48.954141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:48.954141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:48.954141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:48.954141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:02:48.954141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:02:48.954141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:02:48.954141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:02:48.954141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:02:48.954141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:02:48.954141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:02:48.954141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:02:48.954141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 627/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Aug 23 07:02:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:02:48.959628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2ad94-d656-45f1-a85e-885aa5b7177a None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:49.008344 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:49.070957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2ad94-d656-45f1-a85e-885aa5b7177a None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:49.112852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a9f5d-513c-40a3-9b2e-ae624fb06a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:49.113474 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a9f5d-513c-40a3-9b2e-ae624fb06a11 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:49.113753 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a9f5d-513c-40a3-9b2e-ae624fb06a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:49.114071 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a9f5d-513c-40a3-9b2e-ae624fb06a11 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:49.114828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a9f5d-513c-40a3-9b2e-ae624fb06a11 None admin] RBAC: Adding query filter params (name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:49.115119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a9f5d-513c-40a3-9b2e-ae624fb06a11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:49.116576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a9f5d-513c-40a3-9b2e-ae624fb06a11 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:49.125304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 609/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Aug 23 07:02:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:49.131266 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0bd40-b16e-4203-a1bb-5fe7f371e149 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:49.226696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:49.245434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0bd40-b16e-4203-a1bb-5fe7f371e149 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:49.281989 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b98673-c461-40c8-9693-a9e0ee4724e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:49.282623 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b98673-c461-40c8-9693-a9e0ee4724e8 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:49.282900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b98673-c461-40c8-9693-a9e0ee4724e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:49.283234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b98673-c461-40c8-9693-a9e0ee4724e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:49.294351 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b98673-c461-40c8-9693-a9e0ee4724e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:49.295864 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b98673-c461-40c8-9693-a9e0ee4724e8 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:49.311560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-e2b98673-c461-40c8-9693-a9e0ee4724e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:49.311560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:49.311560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:49.311560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:49.311560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:49.311560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:49.311560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:49.311560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:49.311560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:49.311560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:49.311560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:49.311560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:49.311560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:02:49.311560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:02:49.311560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:02:49.311560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:02:49.317700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:49.317700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:49.317700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:49.317700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:49.317700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:49.317700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:49.317700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:49.317700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:49.317700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:49.317700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:49.317700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:49.317700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:49.317700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:49.317700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:49.317700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:49.317700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:49.317700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:02:49.319110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:02:49.319110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:49.319110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:02:49.319110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 628/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:02:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:02:49.319435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a9f5d-513c-40a3-9b2e-ae624fb06a11 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:49.436114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a9f5d-513c-40a3-9b2e-ae624fb06a11 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:49.447083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:49.477246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c166b-f9cb-4d2a-abce-1cccfccb1caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:49.477919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c166b-f9cb-4d2a-abce-1cccfccb1caf None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:49.478194 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c166b-f9cb-4d2a-abce-1cccfccb1caf None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:49.478461 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c166b-f9cb-4d2a-abce-1cccfccb1caf None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:49.479132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c166b-f9cb-4d2a-abce-1cccfccb1caf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:49.479455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c166b-f9cb-4d2a-abce-1cccfccb1caf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:49.480947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c166b-f9cb-4d2a-abce-1cccfccb1caf None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:49.499319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 610/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:49.500522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b98673-c461-40c8-9693-a9e0ee4724e8 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:49.608381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b98673-c461-40c8-9693-a9e0ee4724e8 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:49.646299 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93c839-fce2-497a-ab2e-657a059219d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:49.647002 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93c839-fce2-497a-ab2e-657a059219d3 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:49.647340 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93c839-fce2-497a-ab2e-657a059219d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:49.647662 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93c839-fce2-497a-ab2e-657a059219d3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:49.650792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:49.660075 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93c839-fce2-497a-ab2e-657a059219d3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:49.661524 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93c839-fce2-497a-ab2e-657a059219d3 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:49.673485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-4b93c839-fce2-497a-ab2e-657a059219d3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:02:49.673485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:49.673485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:49.673485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:49.673485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:49.673485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:49.673485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:49.673485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:49.673485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:49.673485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:49.673485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:49.673485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:49.673485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:02:49.673485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:02:49.673485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:02:49.673485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:02:49.674670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:49.674670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:49.674670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:02:49.674670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:02:49.674670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:02:49.674670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:02:49.674670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:49.674670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:49.674670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:49.674670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:49.674670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:49.674670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:49.674670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:49.674670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:49.674670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:49.674670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:49.674670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:49.674670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:49.675642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:49.675642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:49.675642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:02:49.675642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:02:49.675642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:02:49.675642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:02:49.675642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:02:49.675642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:02:49.675642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:02:49.675642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:02:49.675642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 629/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:02:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:02:49.680584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c166b-f9cb-4d2a-abce-1cccfccb1caf None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:49.791911 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c166b-f9cb-4d2a-abce-1cccfccb1caf None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:49.825411 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b965f-016f-40bd-9324-faccd74231a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:49.826076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b965f-016f-40bd-9324-faccd74231a6 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:49.826301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b965f-016f-40bd-9324-faccd74231a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:49.826887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b965f-016f-40bd-9324-faccd74231a6 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:49.827673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b965f-016f-40bd-9324-faccd74231a6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:49.827975 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b965f-016f-40bd-9324-faccd74231a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:49.829431 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b965f-016f-40bd-9324-faccd74231a6 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:49.847807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 611/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Aug 23 07:02:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:02:49.854016 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93c839-fce2-497a-ab2e-657a059219d3 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:49.965836 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93c839-fce2-497a-ab2e-657a059219d3 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:50.000965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdb945-e1d3-4422-8775-f15cd59999c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:50.001586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdb945-e1d3-4422-8775-f15cd59999c5 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:50.001887 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdb945-e1d3-4422-8775-f15cd59999c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:50.002136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdb945-e1d3-4422-8775-f15cd59999c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:50.082514 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:50.090143 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdb945-e1d3-4422-8775-f15cd59999c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:50.091756 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdb945-e1d3-4422-8775-f15cd59999c5 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:50.194881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 630/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:02:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:02:50.201129 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b965f-016f-40bd-9324-faccd74231a6 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:50.231305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:50.310508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b965f-016f-40bd-9324-faccd74231a6 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:50.350282 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d17e4a-ab5a-4e02-abc1-33f9d3f1e5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:50.350828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d17e4a-ab5a-4e02-abc1-33f9d3f1e5ca None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:50.351101 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d17e4a-ab5a-4e02-abc1-33f9d3f1e5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:50.351373 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d17e4a-ab5a-4e02-abc1-33f9d3f1e5ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:50.362054 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d17e4a-ab5a-4e02-abc1-33f9d3f1e5ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:50.363277 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d17e4a-ab5a-4e02-abc1-33f9d3f1e5ca None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:50.378496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-d4d17e4a-ab5a-4e02-abc1-33f9d3f1e5ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:50.378496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:50.378496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:50.378496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:50.378496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:50.378496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:50.378496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:50.378496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:50.378496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:50.378496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:50.378496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:50.378496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:50.378496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:02:50.378496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:02:50.378496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:02:50.378496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:02:50.379637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:50.379637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:50.379637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:50.379637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:50.379637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:50.379637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:50.379637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:50.379637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:50.379637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:50.379637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:50.379637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:50.379637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:50.379637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:50.379637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:50.379637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:50.379637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:50.379637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:02:50.380542 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:02:50.380542 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:50.380542 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:02:50.380542 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 612/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:02:50.385552 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdb945-e1d3-4422-8775-f15cd59999c5 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:50.506878 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:50.515329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdb945-e1d3-4422-8775-f15cd59999c5 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:50.558617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3b693-f102-428e-bcc4-f518ffab6142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:50.559819 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3b693-f102-428e-bcc4-f518ffab6142 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:50.560341 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3b693-f102-428e-bcc4-f518ffab6142 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:50.560878 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3b693-f102-428e-bcc4-f518ffab6142 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:50.561987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3b693-f102-428e-bcc4-f518ffab6142 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:50.562442 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3b693-f102-428e-bcc4-f518ffab6142 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:50.564930 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3b693-f102-428e-bcc4-f518ffab6142 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:50.581351 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 631/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:50.587955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d17e4a-ab5a-4e02-abc1-33f9d3f1e5ca None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:50.676794 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:50.703393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d17e4a-ab5a-4e02-abc1-33f9d3f1e5ca None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:50.741975 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b92c7d-4d28-4df9-aa85-023576df84ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:50.742673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b92c7d-4d28-4df9-aa85-023576df84ae None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:50.742946 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b92c7d-4d28-4df9-aa85-023576df84ae None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:50.743214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b92c7d-4d28-4df9-aa85-023576df84ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:50.750070 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b92c7d-4d28-4df9-aa85-023576df84ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:50.751555 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b92c7d-4d28-4df9-aa85-023576df84ae None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:50.763705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-08b92c7d-4d28-4df9-aa85-023576df84ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:02:50.763705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:50.763705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:50.763705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:50.763705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:50.763705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:50.763705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:50.763705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:50.763705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:50.763705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:50.763705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:50.763705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:50.763705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:02:50.763705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:02:50.763705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:02:50.769624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:02:50.769624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:50.769624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:50.769624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:50.769624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:50.769624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:50.769624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:50.769624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:50.769624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:50.769624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:50.769624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:50.769624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:50.769624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:50.769624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:50.769624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:50.769624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:50.769624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:50.770626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:02:50.770626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:02:50.770626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:02:50.770626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:02:50.770626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:02:50.770626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:02:50.770626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:02:50.770626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:02:50.770626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 613/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:02:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:02:50.771126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3b693-f102-428e-bcc4-f518ffab6142 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:50.880316 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3b693-f102-428e-bcc4-f518ffab6142 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:50.919524 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0e236-2c4f-4d3f-86eb-e37eddaf1b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:50.920082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0e236-2c4f-4d3f-86eb-e37eddaf1b68 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:50.920651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0e236-2c4f-4d3f-86eb-e37eddaf1b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:50.920969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0e236-2c4f-4d3f-86eb-e37eddaf1b68 None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:50.921894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0e236-2c4f-4d3f-86eb-e37eddaf1b68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:50.922234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0e236-2c4f-4d3f-86eb-e37eddaf1b68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:50.924003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0e236-2c4f-4d3f-86eb-e37eddaf1b68 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:50.937646 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 632/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:02:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:50.944065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b92c7d-4d28-4df9-aa85-023576df84ae None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:51.061160 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b92c7d-4d28-4df9-aa85-023576df84ae None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:51.107386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a90700-60b4-481e-979f-44fe18965161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:51.108029 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a90700-60b4-481e-979f-44fe18965161 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:51.108603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a90700-60b4-481e-979f-44fe18965161 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:51.108603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a90700-60b4-481e-979f-44fe18965161 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:51.109401 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a90700-60b4-481e-979f-44fe18965161 None admin] RBAC: Adding query filter params (role.id=2d62b8b8395745c7983c6c8d100d985b, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=a9b0f54c988e43bda474f978a69fa4fe) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:51.109603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a90700-60b4-481e-979f-44fe18965161 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:51.111059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a90700-60b4-481e-979f-44fe18965161 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:51.117739 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 614/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:02:50 2022] GET /identity/v3/role_assignments?user.id=a9b0f54c988e43bda474f978a69fa4fe&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2d62b8b8395745c7983c6c8d100d985b => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:02:53.097719 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dc650-8247-477c-9f15-a00b2c936f90 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:53.098264 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dc650-8247-477c-9f15-a00b2c936f90 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:53.098264 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dc650-8247-477c-9f15-a00b2c936f90 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:53.099574 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 633/1247] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:02:53.105167 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff01f5-8c46-48bf-b799-b1e6e777ae87 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:53.105757 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff01f5-8c46-48bf-b799-b1e6e777ae87 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:53.105757 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff01f5-8c46-48bf-b799-b1e6e777ae87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:53.112247 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:53.126344 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-87ff01f5-8c46-48bf-b799-b1e6e777ae87 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:02:53.132357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-87ff01f5-8c46-48bf-b799-b1e6e777ae87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:02:53.211944 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 615/1248] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:02:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:02:53.500948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068dc650-8247-477c-9f15-a00b2c936f90 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:53.524811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:53.629710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068dc650-8247-477c-9f15-a00b2c936f90 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:53.671258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04294d1c-47af-445b-9bf0-8d2acd3a479e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:53.671964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04294d1c-47af-445b-9bf0-8d2acd3a479e None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:53.672244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04294d1c-47af-445b-9bf0-8d2acd3a479e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:53.672509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04294d1c-47af-445b-9bf0-8d2acd3a479e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:53.684389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04294d1c-47af-445b-9bf0-8d2acd3a479e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:02:53.690020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-04294d1c-47af-445b-9bf0-8d2acd3a479e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:02:53.690020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:53.690020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:53.690020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:53.690020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:53.690020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:53.690020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:53.690020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:53.690020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:53.690020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:53.690020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:53.690020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:53.690020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:02:53.690020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:02:53.690020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:02:53.690020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:02:53.691570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:02:53.691570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:02:53.691570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:02:53.691570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:02:53.691570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:53.691570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:53.691570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:53.691570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:53.691570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:53.691570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:53.691570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:53.691570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:53.691570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:53.691570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:53.691570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:53.691570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:53.692686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:53.692686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:53.692686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:53.692686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:53.692686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:02:53.692686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:02:53.692686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:02:53.692686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:02:53.692686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:02:53.692686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:02:53.692686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:02:53.692686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:02:53.692686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 634/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:02:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:02:53.698071 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff01f5-8c46-48bf-b799-b1e6e777ae87 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:53.827140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff01f5-8c46-48bf-b799-b1e6e777ae87 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:53.861439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:53.866916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd341e-b128-49b8-8f05-b637f2b29ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:53.867566 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd341e-b128-49b8-8f05-b637f2b29ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:53.867849 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd341e-b128-49b8-8f05-b637f2b29ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:53.868124 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd341e-b128-49b8-8f05-b637f2b29ab8 None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:53.868775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd341e-b128-49b8-8f05-b637f2b29ab8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:53.869072 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd341e-b128-49b8-8f05-b637f2b29ab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:53.870447 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd341e-b128-49b8-8f05-b637f2b29ab8 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:53.879055 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 616/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 07:02:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:53.884867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04294d1c-47af-445b-9bf0-8d2acd3a479e None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:53.977330 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:54.002568 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04294d1c-47af-445b-9bf0-8d2acd3a479e None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:54.048795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345f434-2105-4dfc-a920-b27230c41379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:54.049832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345f434-2105-4dfc-a920-b27230c41379 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:54.050282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345f434-2105-4dfc-a920-b27230c41379 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:54.050613 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345f434-2105-4dfc-a920-b27230c41379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:54.066207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345f434-2105-4dfc-a920-b27230c41379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:54.067937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345f434-2105-4dfc-a920-b27230c41379 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:54.089787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-9345f434-2105-4dfc-a920-b27230c41379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:54.089787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:54.089787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:54.089787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:54.089787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:54.089787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:54.089787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:54.089787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:54.089787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:54.089787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:54.089787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:54.089787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:54.089787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:02:54.089787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:02:54.089787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:02:54.089787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:02:54.091155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:54.091155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:54.091155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:54.091155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:54.091155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:54.091155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:54.091155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:54.091155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:54.091155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:54.091155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:54.091155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:54.091155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:54.091155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:54.091155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:54.091155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:54.091155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:54.091155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:02:54.092169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:02:54.092169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:54.092169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:02:54.092169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 635/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:02:54.098903 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd341e-b128-49b8-8f05-b637f2b29ab8 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:54.230415 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd341e-b128-49b8-8f05-b637f2b29ab8 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:54.268220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec335f19-6711-467f-83ae-b9d372f1e90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:54.268559 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec335f19-6711-467f-83ae-b9d372f1e90c None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:54.268788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec335f19-6711-467f-83ae-b9d372f1e90c None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:54.269060 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec335f19-6711-467f-83ae-b9d372f1e90c None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:54.269719 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec335f19-6711-467f-83ae-b9d372f1e90c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:54.270006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec335f19-6711-467f-83ae-b9d372f1e90c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:54.271467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec335f19-6711-467f-83ae-b9d372f1e90c None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:54.284298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 617/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:54.290519 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345f434-2105-4dfc-a920-b27230c41379 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:54.410438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345f434-2105-4dfc-a920-b27230c41379 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:54.438703 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:54.448089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa01bd2-a47f-43a6-a646-c98be075dfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:54.448866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa01bd2-a47f-43a6-a646-c98be075dfc0 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:54.449136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa01bd2-a47f-43a6-a646-c98be075dfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:54.449416 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa01bd2-a47f-43a6-a646-c98be075dfc0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:54.458912 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa01bd2-a47f-43a6-a646-c98be075dfc0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:54.460395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa01bd2-a47f-43a6-a646-c98be075dfc0 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:54.475115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-daa01bd2-a47f-43a6-a646-c98be075dfc0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:02:54.475115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:54.475115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:54.475115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:54.475115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:54.475115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:54.475115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:54.475115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:54.475115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:54.475115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:54.475115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:54.475115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:54.475115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:02:54.475115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:02:54.475115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:02:54.475115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:02:54.476346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:54.476346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:54.476346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:02:54.476346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:02:54.476346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:02:54.476346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:02:54.476346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:54.476346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:54.476346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:54.476346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:54.476346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:54.476346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:54.476346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:54.476346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:54.476346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:54.476346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:54.476346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:54.476346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:54.477456 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:54.477456 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:54.477456 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:02:54.477456 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:02:54.477456 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:02:54.477456 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:02:54.477456 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:02:54.477456 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:02:54.477456 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:02:54.477456 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:02:54.477456 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 636/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:02:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:02:54.482408 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec335f19-6711-467f-83ae-b9d372f1e90c None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:54.550071 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:54.609272 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec335f19-6711-467f-83ae-b9d372f1e90c None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:54.650805 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e23876-1adc-4ce3-a8d0-7a4d09e27486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:54.651439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e23876-1adc-4ce3-a8d0-7a4d09e27486 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:54.651722 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e23876-1adc-4ce3-a8d0-7a4d09e27486 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:54.651993 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e23876-1adc-4ce3-a8d0-7a4d09e27486 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:54.652734 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e23876-1adc-4ce3-a8d0-7a4d09e27486 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:54.653035 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e23876-1adc-4ce3-a8d0-7a4d09e27486 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:54.654624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e23876-1adc-4ce3-a8d0-7a4d09e27486 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:54.673916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 618/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Aug 23 07:02:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:02:54.680345 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa01bd2-a47f-43a6-a646-c98be075dfc0 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:54.788658 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa01bd2-a47f-43a6-a646-c98be075dfc0 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:54.823097 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2d16d-e6a1-40fc-8b12-44aeeafcd728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:54.823758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2d16d-e6a1-40fc-8b12-44aeeafcd728 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:54.824049 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2d16d-e6a1-40fc-8b12-44aeeafcd728 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:54.824303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2d16d-e6a1-40fc-8b12-44aeeafcd728 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:54.889151 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:54.902537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2d16d-e6a1-40fc-8b12-44aeeafcd728 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:54.904281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2d16d-e6a1-40fc-8b12-44aeeafcd728 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:55.011267 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 637/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:02:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:02:55.017285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e23876-1adc-4ce3-a8d0-7a4d09e27486 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:55.132228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e23876-1adc-4ce3-a8d0-7a4d09e27486 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:55.147248 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:55.173656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c0d94-57ba-4538-9329-bdcca51c39eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:55.174083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c0d94-57ba-4538-9329-bdcca51c39eb None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:55.174364 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c0d94-57ba-4538-9329-bdcca51c39eb None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:55.174649 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c0d94-57ba-4538-9329-bdcca51c39eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:55.184659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c0d94-57ba-4538-9329-bdcca51c39eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:55.186124 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c0d94-57ba-4538-9329-bdcca51c39eb None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:55.202795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-f12c0d94-57ba-4538-9329-bdcca51c39eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:55.202795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:55.202795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:55.202795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:55.202795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:55.202795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:55.202795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:55.202795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:55.202795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:55.202795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:55.202795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:55.202795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:55.202795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:02:55.202795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:02:55.202795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:02:55.202795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:02:55.204268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:55.204268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:55.204268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:55.204268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:55.204268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:55.204268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:55.204268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:55.204268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:55.204268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:55.204268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:55.204268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:55.204268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:55.204268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:55.204268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:55.204268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:55.204268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:55.204268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:02:55.205862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:02:55.205862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:55.205862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:02:55.205862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 619/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:02:55.210453 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2d16d-e6a1-40fc-8b12-44aeeafcd728 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:55.313491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:55.326154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2d16d-e6a1-40fc-8b12-44aeeafcd728 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:55.365033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c24f03-e31a-4d04-b723-9bb196cf2da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:55.365657 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c24f03-e31a-4d04-b723-9bb196cf2da3 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:55.365928 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c24f03-e31a-4d04-b723-9bb196cf2da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:55.366174 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c24f03-e31a-4d04-b723-9bb196cf2da3 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:55.366846 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c24f03-e31a-4d04-b723-9bb196cf2da3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:55.367156 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c24f03-e31a-4d04-b723-9bb196cf2da3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:55.368540 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c24f03-e31a-4d04-b723-9bb196cf2da3 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:55.381382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 638/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:55.388287 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c0d94-57ba-4538-9329-bdcca51c39eb None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:55.503170 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c0d94-57ba-4538-9329-bdcca51c39eb None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:55.538753 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccf266-5ba7-45b6-8115-d506f887dd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:55.539367 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccf266-5ba7-45b6-8115-d506f887dd71 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:55.539672 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccf266-5ba7-45b6-8115-d506f887dd71 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:55.539945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccf266-5ba7-45b6-8115-d506f887dd71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:55.546964 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccf266-5ba7-45b6-8115-d506f887dd71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:55.548669 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccf266-5ba7-45b6-8115-d506f887dd71 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:55.560330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-0cccf266-5ba7-45b6-8115-d506f887dd71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:02:55.560330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:55.560330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:55.560330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:55.560330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:55.560330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:55.560330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:55.560330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:55.560330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:55.560330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:55.560330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:55.560330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:55.560330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:02:55.560330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:02:55.560330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:02:55.561437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:02:55.561437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:55.561437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:55.561437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:55.561437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:55.561437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:55.561437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:55.561437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:55.561437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:55.561437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:55.561437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:55.561437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:55.561437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:55.561437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:55.561437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:55.561437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:55.561437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:55.562578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:02:55.562578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:02:55.562578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:02:55.562578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:02:55.562578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:02:55.562578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:02:55.562578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:02:55.562578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:02:55.562578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 620/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:02:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:02:55.567991 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c24f03-e31a-4d04-b723-9bb196cf2da3 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:55.682578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c24f03-e31a-4d04-b723-9bb196cf2da3 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:55.720028 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8b1bc-e9c6-40a5-ae17-d9f929be533e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:55.720834 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8b1bc-e9c6-40a5-ae17-d9f929be533e None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:55.721112 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8b1bc-e9c6-40a5-ae17-d9f929be533e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:55.721390 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8b1bc-e9c6-40a5-ae17-d9f929be533e None admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:55.722081 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8b1bc-e9c6-40a5-ae17-d9f929be533e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:55.722382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8b1bc-e9c6-40a5-ae17-d9f929be533e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:55.723851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8b1bc-e9c6-40a5-ae17-d9f929be533e None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:55.735802 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 639/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:02:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:02:55.743280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccf266-5ba7-45b6-8115-d506f887dd71 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:55.773000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:55.856262 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccf266-5ba7-45b6-8115-d506f887dd71 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:55.892396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826a337-f4d3-4cda-ba23-568fe1691408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:55.893083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826a337-f4d3-4cda-ba23-568fe1691408 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:55.893392 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826a337-f4d3-4cda-ba23-568fe1691408 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:55.893699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826a337-f4d3-4cda-ba23-568fe1691408 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:55.894528 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826a337-f4d3-4cda-ba23-568fe1691408 None admin] RBAC: Adding query filter params (role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=a9b0f54c988e43bda474f978a69fa4fe) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:55.894845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826a337-f4d3-4cda-ba23-568fe1691408 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:55.896597 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826a337-f4d3-4cda-ba23-568fe1691408 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:55.914722 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 621/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:02:55 2022] GET /identity/v3/role_assignments?user.id=a9b0f54c988e43bda474f978a69fa4fe&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:57.823683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6b438-667a-4c02-a241-eb1de5bc183f None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:57.824363 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6b438-667a-4c02-a241-eb1de5bc183f None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:57.824363 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6b438-667a-4c02-a241-eb1de5bc183f None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:57.826831 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 640/1261] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:02:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:02:57.834662 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bd372-eb0a-4b28-a333-6220d8787820 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:57.835128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bd372-eb0a-4b28-a333-6220d8787820 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:57.835221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bd372-eb0a-4b28-a333-6220d8787820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:57.855491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-481bd372-eb0a-4b28-a333-6220d8787820 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:02:57.856684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-481bd372-eb0a-4b28-a333-6220d8787820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:02:57.928716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 622/1262] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:02:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:02:58.192554 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6b438-667a-4c02-a241-eb1de5bc183f None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:58.205577 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:58.321725 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6b438-667a-4c02-a241-eb1de5bc183f None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:58.364495 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6fb69-3046-46ac-b6d0-87c2fd7b6765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:58.365107 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6fb69-3046-46ac-b6d0-87c2fd7b6765 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:58.365379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6fb69-3046-46ac-b6d0-87c2fd7b6765 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:58.365655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6fb69-3046-46ac-b6d0-87c2fd7b6765 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:58.376288 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6fb69-3046-46ac-b6d0-87c2fd7b6765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:02:58.381189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-44a6fb69-3046-46ac-b6d0-87c2fd7b6765 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:02:58.381189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:58.381189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:58.381189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:58.381189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:58.381189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:58.381189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:58.381189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:58.381189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:58.381189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:58.381189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:58.381189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:58.381189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:02:58.381189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:02:58.381189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:02:58.381189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:02:58.382271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:02:58.382271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:02:58.382271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:02:58.382271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:02:58.382271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:58.382271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:58.382271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:58.382271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:58.382271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:58.382271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:58.382271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:58.382271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:58.382271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:58.382271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:58.382271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:58.382271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:58.383227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:58.383227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:58.383227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:58.383227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:58.383227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:02:58.383227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:02:58.383227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:02:58.383227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:02:58.383227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:02:58.383227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:02:58.383227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:02:58.383227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:02:58.383227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 641/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:02:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:02:58.388489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bd372-eb0a-4b28-a333-6220d8787820 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:58.408501 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:58.490700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bd372-eb0a-4b28-a333-6220d8787820 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:58.530762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b24c9-18c7-4bc9-8aec-f72554116b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:58.531354 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b24c9-18c7-4bc9-8aec-f72554116b1d None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:58.531660 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b24c9-18c7-4bc9-8aec-f72554116b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:58.531923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b24c9-18c7-4bc9-8aec-f72554116b1d None admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:58.532602 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b24c9-18c7-4bc9-8aec-f72554116b1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:58.533020 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b24c9-18c7-4bc9-8aec-f72554116b1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:58.534394 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b24c9-18c7-4bc9-8aec-f72554116b1d None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:58.542384 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 623/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 07:02:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:58.548595 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6fb69-3046-46ac-b6d0-87c2fd7b6765 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:58.629948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:58.676421 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6fb69-3046-46ac-b6d0-87c2fd7b6765 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:58.717706 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325b80e-95d1-4542-86b8-4f70610ba4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:58.718600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325b80e-95d1-4542-86b8-4f70610ba4ec None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:58.718955 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325b80e-95d1-4542-86b8-4f70610ba4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:58.719306 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325b80e-95d1-4542-86b8-4f70610ba4ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:58.730445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325b80e-95d1-4542-86b8-4f70610ba4ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:58.732019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325b80e-95d1-4542-86b8-4f70610ba4ec None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:58.746853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-f325b80e-95d1-4542-86b8-4f70610ba4ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:58.746853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:58.746853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:58.746853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:58.746853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:58.746853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:58.746853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:58.746853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:58.746853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:58.746853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:58.746853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:58.746853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:58.746853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:02:58.746853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:02:58.746853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:02:58.746853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:02:58.747964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:58.747964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:58.747964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:58.747964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:58.747964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:58.747964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:58.747964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:58.747964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:58.747964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:58.747964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:58.747964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:58.747964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:58.747964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:58.747964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:58.747964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:58.747964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:58.747964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:02:58.749184 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:02:58.749184 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:58.749184 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:02:58.749184 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 642/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:02:58.754560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b24c9-18c7-4bc9-8aec-f72554116b1d None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:58.854450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:58.878246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b24c9-18c7-4bc9-8aec-f72554116b1d None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:58.914316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f852afb-39f4-4ae0-890f-7f521d698ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:58.914910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f852afb-39f4-4ae0-890f-7f521d698ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:58.915188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f852afb-39f4-4ae0-890f-7f521d698ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:58.915441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f852afb-39f4-4ae0-890f-7f521d698ba0 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:58.916118 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f852afb-39f4-4ae0-890f-7f521d698ba0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:58.916422 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f852afb-39f4-4ae0-890f-7f521d698ba0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:58.917787 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f852afb-39f4-4ae0-890f-7f521d698ba0 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:58.929824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 624/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:58.938890 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325b80e-95d1-4542-86b8-4f70610ba4ec None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:59.055869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325b80e-95d1-4542-86b8-4f70610ba4ec None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:59.079081 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:59.097434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cbdab-44c4-495d-b199-b84e59bd916d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:59.098200 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cbdab-44c4-495d-b199-b84e59bd916d None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:59.098417 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cbdab-44c4-495d-b199-b84e59bd916d None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:59.098658 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cbdab-44c4-495d-b199-b84e59bd916d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:59.108548 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716cbdab-44c4-495d-b199-b84e59bd916d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:59.110128 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716cbdab-44c4-495d-b199-b84e59bd916d None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:59.123071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-716cbdab-44c4-495d-b199-b84e59bd916d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:02:59.123071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:59.123071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:59.123071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:59.123071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:59.123071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:59.123071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:59.123071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:59.123071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:59.123071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:59.123071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:59.123071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:59.123071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:02:59.123071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:02:59.123071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:02:59.123071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:02:59.124235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:59.124235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:59.124235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:02:59.124235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:02:59.124235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:02:59.124235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:02:59.124235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:59.124235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:59.124235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:59.124235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:59.124235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:59.124235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:59.124235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:59.124235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:59.124235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:59.124235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:59.124235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:59.124235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:59.125561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:59.125561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:59.125561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:02:59.125561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:02:59.125561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:02:59.125561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:02:59.125561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:02:59.125561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:02:59.125561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:02:59.125561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:02:59.125561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 643/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:02:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:02:59.130352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f852afb-39f4-4ae0-890f-7f521d698ba0 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:59.237229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f852afb-39f4-4ae0-890f-7f521d698ba0 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:59.272380 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246bfab-90bf-4774-bfcc-c8ca353194d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:59.272986 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246bfab-90bf-4774-bfcc-c8ca353194d1 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:59.273263 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246bfab-90bf-4774-bfcc-c8ca353194d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:59.273542 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246bfab-90bf-4774-bfcc-c8ca353194d1 None admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:59.274269 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246bfab-90bf-4774-bfcc-c8ca353194d1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:59.274579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246bfab-90bf-4774-bfcc-c8ca353194d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:59.276092 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246bfab-90bf-4774-bfcc-c8ca353194d1 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:59.285104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:59.294071 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 625/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Aug 23 07:02:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:59.300843 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cbdab-44c4-495d-b199-b84e59bd916d None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:59.411554 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cbdab-44c4-495d-b199-b84e59bd916d None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:59.453866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e40b40-5c92-436c-851f-105ac2edebc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:59.454544 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e40b40-5c92-436c-851f-105ac2edebc9 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:59.454933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e40b40-5c92-436c-851f-105ac2edebc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:59.455207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e40b40-5c92-436c-851f-105ac2edebc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:59.466452 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e40b40-5c92-436c-851f-105ac2edebc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:59.467937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e40b40-5c92-436c-851f-105ac2edebc9 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:59.482823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-63e40b40-5c92-436c-851f-105ac2edebc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:59.482823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:59.482823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:59.482823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:59.482823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:59.482823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:59.482823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:59.482823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:59.482823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:59.482823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:59.482823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:59.482823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:59.482823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:02:59.482823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:02:59.482823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:02:59.482823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:02:59.485980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:59.485980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:59.485980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:59.485980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:59.485980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:59.485980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:59.485980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:59.485980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:59.485980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:59.485980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:59.485980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:59.485980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:59.485980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:59.485980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:59.485980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:59.485980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:59.485980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:02:59.487879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:02:59.487879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:02:59.487879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:02:59.487879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 644/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:02:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:02:59.492979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246bfab-90bf-4774-bfcc-c8ca353194d1 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:59.616100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246bfab-90bf-4774-bfcc-c8ca353194d1 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:59.659732 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f775f5c-fe3c-4e36-909e-50e00dd3954a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:59.660628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f775f5c-fe3c-4e36-909e-50e00dd3954a None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:59.661000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f775f5c-fe3c-4e36-909e-50e00dd3954a None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:59.661365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f775f5c-fe3c-4e36-909e-50e00dd3954a None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:59.662435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f775f5c-fe3c-4e36-909e-50e00dd3954a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:02:59.662733 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f775f5c-fe3c-4e36-909e-50e00dd3954a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:59.664947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f775f5c-fe3c-4e36-909e-50e00dd3954a None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:59.680177 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 626/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:02:59.686183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e40b40-5c92-436c-851f-105ac2edebc9 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:59.722211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:59.804411 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e40b40-5c92-436c-851f-105ac2edebc9 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:02:59.839917 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99c00f-fcec-4c07-9400-5ef02448f5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:02:59.840518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99c00f-fcec-4c07-9400-5ef02448f5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:02:59.840790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99c00f-fcec-4c07-9400-5ef02448f5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:02:59.841059 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99c00f-fcec-4c07-9400-5ef02448f5a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:02:59.848195 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99c00f-fcec-4c07-9400-5ef02448f5a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:02:59.849629 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99c00f-fcec-4c07-9400-5ef02448f5a1 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:02:59.860870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-1d99c00f-fcec-4c07-9400-5ef02448f5a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:02:59.860870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:02:59.860870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:02:59.860870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:02:59.860870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:02:59.860870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:02:59.860870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:02:59.860870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:02:59.860870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:02:59.860870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:02:59.860870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:02:59.860870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:02:59.860870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:02:59.860870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:02:59.860870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:02:59.862024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:02:59.862024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:02:59.862024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:02:59.862024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:02:59.862024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:02:59.862024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:02:59.862024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:02:59.862024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:02:59.862024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:02:59.862024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:02:59.862024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:02:59.862024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:02:59.862024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:02:59.862024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:02:59.862024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:02:59.862024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:02:59.862024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:02:59.863630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:02:59.863630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:02:59.863630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:02:59.863630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:02:59.863630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:02:59.863630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:02:59.863630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:02:59.863630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:02:59.863630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 645/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:02:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:02:59.871659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f775f5c-fe3c-4e36-909e-50e00dd3954a None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:02:59.943194 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:02:59.988357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f775f5c-fe3c-4e36-909e-50e00dd3954a None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:00.022901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6930c6-6abe-40ed-857b-09fcd6b1a154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:00.023525 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6930c6-6abe-40ed-857b-09fcd6b1a154 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:00.023795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6930c6-6abe-40ed-857b-09fcd6b1a154 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:00.024066 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6930c6-6abe-40ed-857b-09fcd6b1a154 None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:00.024753 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6930c6-6abe-40ed-857b-09fcd6b1a154 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:03:00.025055 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6930c6-6abe-40ed-857b-09fcd6b1a154 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:00.026501 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6930c6-6abe-40ed-857b-09fcd6b1a154 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:00.038658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 627/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:02:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:03:00.046457 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99c00f-fcec-4c07-9400-5ef02448f5a1 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:00.158277 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:00.170190 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99c00f-fcec-4c07-9400-5ef02448f5a1 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:00.207035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62967777-1e35-4976-8e4e-5f4fed225b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:00.207642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62967777-1e35-4976-8e4e-5f4fed225b3e None admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:00.207918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62967777-1e35-4976-8e4e-5f4fed225b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:00.208192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62967777-1e35-4976-8e4e-5f4fed225b3e None admin] PATH_INFO: `/v3/projects/351a6777525c4207826d7df0d3768e99/users/a9b0f54c988e43bda474f978a69fa4fe/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:00.210216 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62967777-1e35-4976-8e4e-5f4fed225b3e None admin] RBAC: Authorizing `identity:create_grant(project_id=351a6777525c4207826d7df0d3768e99, user_id=a9b0f54c988e43bda474f978a69fa4fe, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:00.211698 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62967777-1e35-4976-8e4e-5f4fed225b3e None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:00.226511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 646/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Aug 23 07:03:00 2022] PUT /identity/v3/projects/351a6777525c4207826d7df0d3768e99/users/a9b0f54c988e43bda474f978a69fa4fe/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:03:02.143003 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994fb577-fae6-4223-b521-095d8833417a None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:02.143754 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994fb577-fae6-4223-b521-095d8833417a None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:02.143754 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994fb577-fae6-4223-b521-095d8833417a None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:02.145580 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 628/1274] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:03:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:03:02.151177 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d70ba6-68f4-4bf0-bb82-4e3956fcf7e3 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:02.151662 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d70ba6-68f4-4bf0-bb82-4e3956fcf7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:02.151773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d70ba6-68f4-4bf0-bb82-4e3956fcf7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:02.173245 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-16d70ba6-68f4-4bf0-bb82-4e3956fcf7e3 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:03:02.175920 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-16d70ba6-68f4-4bf0-bb82-4e3956fcf7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:03:02.263555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 647/1275] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:03:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:03:02.554079 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994fb577-fae6-4223-b521-095d8833417a None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:02.683787 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994fb577-fae6-4223-b521-095d8833417a None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:02.705314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:02.724180 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5448bec-fcef-417e-9b93-e04841a7ce62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:02.725248 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5448bec-fcef-417e-9b93-e04841a7ce62 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:02.725715 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5448bec-fcef-417e-9b93-e04841a7ce62 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:02.726002 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5448bec-fcef-417e-9b93-e04841a7ce62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:02.738139 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5448bec-fcef-417e-9b93-e04841a7ce62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:03:02.742770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-e5448bec-fcef-417e-9b93-e04841a7ce62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:03:02.742770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:03:02.742770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:03:02.742770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:03:02.742770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:03:02.742770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:03:02.742770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:03:02.742770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:03:02.742770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:03:02.742770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:03:02.742770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:03:02.742770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:03:02.742770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:03:02.742770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:03:02.742770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:03:02.742770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:03:02.743915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:03:02.743915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:03:02.743915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:03:02.743915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:03:02.743915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:03:02.743915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:03:02.743915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:03:02.743915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:03:02.743915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:03:02.743915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:03:02.743915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:03:02.743915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:03:02.743915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:03:02.743915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:03:02.743915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:03:02.743915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:03:02.744994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:03:02.744994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:03:02.744994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:03:02.744994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:03:02.744994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:03:02.744994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:03:02.744994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:03:02.744994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:03:02.744994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:03:02.744994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:03:02.744994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:03:02.744994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:03:02.744994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 629/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Aug 23 07:03:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:03:02.751979 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d70ba6-68f4-4bf0-bb82-4e3956fcf7e3 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:02.844990 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:02.864563 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d70ba6-68f4-4bf0-bb82-4e3956fcf7e3 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:02.903459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423f1c5-9a6a-49b8-9b6b-1d01a82024a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:02.904523 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423f1c5-9a6a-49b8-9b6b-1d01a82024a1 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:02.905024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423f1c5-9a6a-49b8-9b6b-1d01a82024a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:02.905522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423f1c5-9a6a-49b8-9b6b-1d01a82024a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:02.906503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423f1c5-9a6a-49b8-9b6b-1d01a82024a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:03:02.906805 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423f1c5-9a6a-49b8-9b6b-1d01a82024a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:02.908323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423f1c5-9a6a-49b8-9b6b-1d01a82024a1 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:02.917096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 648/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Aug 23 07:03:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:03:02.922987 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5448bec-fcef-417e-9b93-e04841a7ce62 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:03.027422 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5448bec-fcef-417e-9b93-e04841a7ce62 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:03.076989 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b4c96-b58b-4675-b324-c85ab9c30e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:03.077661 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b4c96-b58b-4675-b324-c85ab9c30e63 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:03.078008 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b4c96-b58b-4675-b324-c85ab9c30e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:03.078350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b4c96-b58b-4675-b324-c85ab9c30e63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:03.091157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b4c96-b58b-4675-b324-c85ab9c30e63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:03.092879 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b4c96-b58b-4675-b324-c85ab9c30e63 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:03.111571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-ba5b4c96-b58b-4675-b324-c85ab9c30e63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:03:03.111571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:03:03.111571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:03:03.111571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:03:03.111571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:03:03.111571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:03:03.111571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:03:03.111571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:03:03.111571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:03:03.111571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:03:03.111571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:03:03.111571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:03:03.111571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:03:03.111571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:03:03.111571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:03:03.111571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:03:03.112738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:03:03.112738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:03:03.112738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:03:03.112738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:03:03.112738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:03:03.112738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:03:03.112738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:03:03.112738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:03:03.112738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:03:03.112738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:03:03.112738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:03:03.112738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:03:03.112738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:03:03.112738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:03:03.112738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:03:03.112738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:03:03.112738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:03:03.113828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:03:03.113828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:03:03.113828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:03:03.113828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 630/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:03:03.119803 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423f1c5-9a6a-49b8-9b6b-1d01a82024a1 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:03.228660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423f1c5-9a6a-49b8-9b6b-1d01a82024a1 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:03.267437 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de417d3-abc3-4a68-b5ea-0ab145137234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:03.268045 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de417d3-abc3-4a68-b5ea-0ab145137234 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:03.268294 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de417d3-abc3-4a68-b5ea-0ab145137234 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:03.268592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de417d3-abc3-4a68-b5ea-0ab145137234 None admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:03.269231 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de417d3-abc3-4a68-b5ea-0ab145137234 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:03:03.269531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de417d3-abc3-4a68-b5ea-0ab145137234 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:03.270893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de417d3-abc3-4a68-b5ea-0ab145137234 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:03.282767 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 649/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:03:03.288707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b4c96-b58b-4675-b324-c85ab9c30e63 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:03.320639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:03.392322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b4c96-b58b-4675-b324-c85ab9c30e63 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:03.428695 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb5297-82a7-4eea-9e77-d284b0c70577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:03.429307 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb5297-82a7-4eea-9e77-d284b0c70577 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:03.429581 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb5297-82a7-4eea-9e77-d284b0c70577 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:03.429850 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb5297-82a7-4eea-9e77-d284b0c70577 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:03.437368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb5297-82a7-4eea-9e77-d284b0c70577 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:03.439129 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb5297-82a7-4eea-9e77-d284b0c70577 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:03.450877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-8ceb5297-82a7-4eea-9e77-d284b0c70577 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:03:03.450877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:03:03.450877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:03:03.450877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:03:03.450877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:03:03.450877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:03:03.450877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:03:03.450877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:03:03.450877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:03:03.450877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:03:03.450877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:03:03.450877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:03:03.450877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:03:03.450877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:03:03.450877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:03:03.450877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:03:03.452343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:03:03.452343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:03:03.452343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:03:03.452343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:03:03.452343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:03:03.452343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:03:03.452343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:03:03.452343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:03:03.452343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:03:03.452343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:03:03.452343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:03:03.452343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:03:03.452343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:03:03.452343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:03:03.452343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:03:03.452343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:03:03.452343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:03:03.452343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:03:03.453532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:03:03.453532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:03:03.453532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:03:03.453532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:03:03.453532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:03:03.453532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:03:03.453532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:03:03.453532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:03:03.453532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:03:03.453532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:03:03.453532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 631/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:03:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:03:03.458928 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de417d3-abc3-4a68-b5ea-0ab145137234 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:03.464801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:03.567941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de417d3-abc3-4a68-b5ea-0ab145137234 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:03.603794 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ddd02-c4ce-45f8-aee3-4f3f06b6a13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:03.604391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ddd02-c4ce-45f8-aee3-4f3f06b6a13c None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:03.604666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ddd02-c4ce-45f8-aee3-4f3f06b6a13c None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:03.604938 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ddd02-c4ce-45f8-aee3-4f3f06b6a13c None admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:03.605656 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ddd02-c4ce-45f8-aee3-4f3f06b6a13c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:03:03.605960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ddd02-c4ce-45f8-aee3-4f3f06b6a13c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:03.607391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ddd02-c4ce-45f8-aee3-4f3f06b6a13c None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:03.624239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 650/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Aug 23 07:03:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:03:03.630548 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb5297-82a7-4eea-9e77-d284b0c70577 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:03.742913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:03.757777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb5297-82a7-4eea-9e77-d284b0c70577 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:03.796556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad4880e-2f4e-48c6-adeb-7d63b1ef9e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:03.797149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4880e-2f4e-48c6-adeb-7d63b1ef9e91 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:03.797426 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4880e-2f4e-48c6-adeb-7d63b1ef9e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:03.797705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4880e-2f4e-48c6-adeb-7d63b1ef9e91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:03.873207 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad4880e-2f4e-48c6-adeb-7d63b1ef9e91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:03.874675 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad4880e-2f4e-48c6-adeb-7d63b1ef9e91 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:03.967701 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:03.980164 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 632/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Aug 23 07:03:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:03:03.988537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ddd02-c4ce-45f8-aee3-4f3f06b6a13c None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:04.056561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:04.104927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ddd02-c4ce-45f8-aee3-4f3f06b6a13c None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:04.140341 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e70b9-ef8f-4824-9fab-320dce7da5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:04.140959 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e70b9-ef8f-4824-9fab-320dce7da5b2 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:04.141232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e70b9-ef8f-4824-9fab-320dce7da5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:04.141501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e70b9-ef8f-4824-9fab-320dce7da5b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:04.151527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e70b9-ef8f-4824-9fab-320dce7da5b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:04.152973 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e70b9-ef8f-4824-9fab-320dce7da5b2 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:04.167816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-341e70b9-ef8f-4824-9fab-320dce7da5b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:03:04.167816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:03:04.167816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:03:04.167816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:03:04.167816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:03:04.167816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:03:04.167816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:03:04.167816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:03:04.167816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:03:04.167816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:03:04.167816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:03:04.167816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:03:04.167816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:03:04.167816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:03:04.167816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:03:04.167816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:03:04.168949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:03:04.168949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:03:04.168949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:03:04.168949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:03:04.168949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:03:04.168949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:03:04.168949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:03:04.168949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:03:04.168949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:03:04.168949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:03:04.168949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:03:04.168949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:03:04.168949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:03:04.168949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:03:04.168949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:03:04.168949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:03:04.168949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:03:04.170148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:03:04.170148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:03:04.170148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:03:04.170148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 651/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:03:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:03:04.174920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad4880e-2f4e-48c6-adeb-7d63b1ef9e91 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:04.274798 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad4880e-2f4e-48c6-adeb-7d63b1ef9e91 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:04.315224 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da508986-30ee-4300-9656-eab1479dc589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:04.315883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da508986-30ee-4300-9656-eab1479dc589 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:04.316174 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da508986-30ee-4300-9656-eab1479dc589 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:04.316550 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da508986-30ee-4300-9656-eab1479dc589 None admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:04.317500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da508986-30ee-4300-9656-eab1479dc589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:03:04.317914 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da508986-30ee-4300-9656-eab1479dc589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:04.320101 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da508986-30ee-4300-9656-eab1479dc589 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:04.335994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 633/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Aug 23 07:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:03:04.341935 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e70b9-ef8f-4824-9fab-320dce7da5b2 None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:04.457650 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e70b9-ef8f-4824-9fab-320dce7da5b2 None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:04.471160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:04.493303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd37060-d599-4ab4-aad8-cf072cd96b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:04.493918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd37060-d599-4ab4-aad8-cf072cd96b4e None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:04.494188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd37060-d599-4ab4-aad8-cf072cd96b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:04.494455 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd37060-d599-4ab4-aad8-cf072cd96b4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:04.501683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd37060-d599-4ab4-aad8-cf072cd96b4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:04.503117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd37060-d599-4ab4-aad8-cf072cd96b4e None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:04.515160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application [None req-9dd37060-d599-4ab4-aad8-cf072cd96b4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:03:04.515160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:03:04.515160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:03:04.515160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:03:04.515160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:03:04.515160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:03:04.515160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:03:04.515160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:03:04.515160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:03:04.515160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:03:04.515160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:03:04.515160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:03:04.515160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:03:04.515160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:03:04.515160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:03:04.516217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:03:04.516217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:03:04.516217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:03:04.516217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:03:04.516217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:03:04.516217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:03:04.516217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:03:04.516217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:03:04.516217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application with Lock( Aug 23 07:03:04.516217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:03:04.516217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:03:04.516217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:03:04.516217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:03:04.516217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:03:04.516217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:03:04.516217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:03:04.516217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:03:04.517641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:03:04.517641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:03:04.517641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:03:04.517641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:03:04.517641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:03:04.517641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:03:04.517641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:03:04.517641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: ERROR keystone.server.flask.application Aug 23 07:03:04.517641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 652/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Aug 23 07:03:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:03:04.522278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da508986-30ee-4300-9656-eab1479dc589 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:04.573865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:04.630375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da508986-30ee-4300-9656-eab1479dc589 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:04.665776 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940df7f2-ee1f-49cb-b9e4-a1cc33c1984b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:04.666379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940df7f2-ee1f-49cb-b9e4-a1cc33c1984b None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:04.666656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940df7f2-ee1f-49cb-b9e4-a1cc33c1984b None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:04.666919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940df7f2-ee1f-49cb-b9e4-a1cc33c1984b None admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:04.667640 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940df7f2-ee1f-49cb-b9e4-a1cc33c1984b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:03:04.667938 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940df7f2-ee1f-49cb-b9e4-a1cc33c1984b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:04.669377 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940df7f2-ee1f-49cb-b9e4-a1cc33c1984b None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:04.681365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 634/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Aug 23 07:03:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:03:04.687666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd37060-d599-4ab4-aad8-cf072cd96b4e None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:04.792511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd37060-d599-4ab4-aad8-cf072cd96b4e None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:04.831505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6e213-e802-43e0-a36b-4a91f03923b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:04.832255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6e213-e802-43e0-a36b-4a91f03923b2 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:04.832529 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6e213-e802-43e0-a36b-4a91f03923b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:04.832797 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6e213-e802-43e0-a36b-4a91f03923b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:04.833506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6e213-e802-43e0-a36b-4a91f03923b2 None admin] RBAC: Adding query filter params (role.id=2fda19ac4eb940169d7e1ef16c60a45f, scope.project.id=351a6777525c4207826d7df0d3768e99, user.id=a9b0f54c988e43bda474f978a69fa4fe) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:03:04.833811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6e213-e802-43e0-a36b-4a91f03923b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:04.835229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6e213-e802-43e0-a36b-4a91f03923b2 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:04.841634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 653/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Aug 23 07:03:04 2022] GET /identity/v3/role_assignments?user.id=a9b0f54c988e43bda474f978a69fa4fe&scope.project.id=351a6777525c4207826d7df0d3768e99&role.id=2fda19ac4eb940169d7e1ef16c60a45f => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:03:06.728948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b4abd-1ab4-45bc-8ea1-652e1a874184 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:06.729758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b4abd-1ab4-45bc-8ea1-652e1a874184 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:06.729938 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b4abd-1ab4-45bc-8ea1-652e1a874184 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:06.732151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 635/1288] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:03:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:03:06.737739 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef5d9c-f029-4a2b-9368-1b09a7ca2f9b None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:06.738176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef5d9c-f029-4a2b-9368-1b09a7ca2f9b None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:06.738282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef5d9c-f029-4a2b-9368-1b09a7ca2f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:06.757690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-e9ef5d9c-f029-4a2b-9368-1b09a7ca2f9b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:03:06.758857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e9ef5d9c-f029-4a2b-9368-1b09a7ca2f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:03:06.789504 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:06.828264 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 654/1289] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:03:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:03:07.096295 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b4abd-1ab4-45bc-8ea1-652e1a874184 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:07.207983 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b4abd-1ab4-45bc-8ea1-652e1a874184 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:07.220281 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:07.246670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c76c5-b836-4d69-994a-cc119403ce43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:07.247320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c76c5-b836-4d69-994a-cc119403ce43 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:07.247674 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c76c5-b836-4d69-994a-cc119403ce43 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:07.247942 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c76c5-b836-4d69-994a-cc119403ce43 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:07.248517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c76c5-b836-4d69-994a-cc119403ce43 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:07.249908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c76c5-b836-4d69-994a-cc119403ce43 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:07.259087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-f73c76c5-b836-4d69-994a-cc119403ce43 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:03:07.259087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:03:07.259087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:03:07.259087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:03:07.259087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:03:07.259087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:03:07.259087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:03:07.259087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:03:07.259087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:03:07.259087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:03:07.259087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:03:07.259087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:03:07.259087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:03:07.259087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:03:07.259087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:03:07.259087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:03:07.259087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:03:07.260313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:03:07.260313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:03:07.260313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:03:07.260313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:03:07.260313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:03:07.260313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:03:07.260313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:03:07.260313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:03:07.260313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:03:07.260313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:03:07.260313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:03:07.260313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:03:07.260313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:03:07.260313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:03:07.260313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:03:07.260313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:03:07.261325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:03:07.261325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:03:07.261325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:03:07.261325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:03:07.261325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:03:07.261325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:03:07.261325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:03:07.261325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:03:07.261325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:03:07.261325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:03:07.261325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:03:07.261325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:03:07.261325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:03:07.261325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:03:07.261325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 636/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 23 07:03:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:03:07.270276 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef5d9c-f029-4a2b-9368-1b09a7ca2f9b None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:07.368810 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef5d9c-f029-4a2b-9368-1b09a7ca2f9b None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:07.404197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae3c99-d41e-4d18-8fce-baff1838a534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:07.404785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae3c99-d41e-4d18-8fce-baff1838a534 None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:07.405059 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae3c99-d41e-4d18-8fce-baff1838a534 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:07.405323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae3c99-d41e-4d18-8fce-baff1838a534 None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:07.405947 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae3c99-d41e-4d18-8fce-baff1838a534 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:03:07.406236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae3c99-d41e-4d18-8fce-baff1838a534 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:07.407833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae3c99-d41e-4d18-8fce-baff1838a534 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:07.412734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 655/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Aug 23 07:03:07 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:03:07.418606 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c76c5-b836-4d69-994a-cc119403ce43 None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:07.528450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c76c5-b836-4d69-994a-cc119403ce43 None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:07.563999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018f678-81bb-41f3-86c5-8245539955a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:07.564598 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018f678-81bb-41f3-86c5-8245539955a2 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:07.564881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018f678-81bb-41f3-86c5-8245539955a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:07.565153 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018f678-81bb-41f3-86c5-8245539955a2 None admin] PATH_INFO: `/v3/services` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:07.565788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018f678-81bb-41f3-86c5-8245539955a2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:03:07.566094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018f678-81bb-41f3-86c5-8245539955a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:07.567475 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018f678-81bb-41f3-86c5-8245539955a2 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:07.572227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 637/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Aug 23 07:03:07 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:03:09.464506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b466db-484a-4656-8e1f-564badd9e3ec None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:09.465370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b466db-484a-4656-8e1f-564badd9e3ec None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:09.465370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b466db-484a-4656-8e1f-564badd9e3ec None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:09.467486 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 656/1293] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:03:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:03:09.474755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23da9fa-6357-4328-9e2e-cd6b072d932e None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:09.475204 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23da9fa-6357-4328-9e2e-cd6b072d932e None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:09.475352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23da9fa-6357-4328-9e2e-cd6b072d932e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:09.499687 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-e23da9fa-6357-4328-9e2e-cd6b072d932e None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:03:09.500967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e23da9fa-6357-4328-9e2e-cd6b072d932e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:03:09.534783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:09.576813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 638/1294] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:03:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:03:09.839897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b466db-484a-4656-8e1f-564badd9e3ec None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:09.874868 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:09.961125 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b466db-484a-4656-8e1f-564badd9e3ec None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:10.004937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82ee37-fc1e-4428-af1d-e6f4abad8a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:10.005541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82ee37-fc1e-4428-af1d-e6f4abad8a86 None admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:10.005817 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82ee37-fc1e-4428-af1d-e6f4abad8a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:10.006087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82ee37-fc1e-4428-af1d-e6f4abad8a86 None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:10.006729 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82ee37-fc1e-4428-af1d-e6f4abad8a86 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:10.008167 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82ee37-fc1e-4428-af1d-e6f4abad8a86 None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:10.024881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 657/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Aug 23 07:03:09 2022] POST /identity/v3/services => generated 246 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:03:11.917950 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0eb790-4d53-461a-8fae-6a0b74080303 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:11.918783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0eb790-4d53-461a-8fae-6a0b74080303 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:11.918783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0eb790-4d53-461a-8fae-6a0b74080303 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:11.920873 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 639/1296] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:03:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:03:11.929770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5bf5f4-f210-4762-9983-b32fdae0b49d None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:11.930181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5bf5f4-f210-4762-9983-b32fdae0b49d None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:11.930398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5bf5f4-f210-4762-9983-b32fdae0b49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:11.952297 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-4a5bf5f4-f210-4762-9983-b32fdae0b49d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:03:11.953496 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4a5bf5f4-f210-4762-9983-b32fdae0b49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:03:12.029749 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 658/1297] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:03:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:03:12.295949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0eb790-4d53-461a-8fae-6a0b74080303 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:12.425730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0eb790-4d53-461a-8fae-6a0b74080303 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:12.462707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf469d-eccc-4de0-aea4-57f5fe39b11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:12.463324 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf469d-eccc-4de0-aea4-57f5fe39b11b None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:12.463632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf469d-eccc-4de0-aea4-57f5fe39b11b None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:12.463901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf469d-eccc-4de0-aea4-57f5fe39b11b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:12.464480 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf469d-eccc-4de0-aea4-57f5fe39b11b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:12.465853 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf469d-eccc-4de0-aea4-57f5fe39b11b None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:12.474779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-8ccf469d-eccc-4de0-aea4-57f5fe39b11b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:03:12.474779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:03:12.474779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:03:12.474779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:03:12.474779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:03:12.474779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:03:12.474779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:03:12.474779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:03:12.474779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:03:12.474779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:03:12.474779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:03:12.474779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:03:12.474779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:03:12.474779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:03:12.474779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:03:12.474779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:03:12.474779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:03:12.476000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:03:12.476000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:03:12.476000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:03:12.476000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:03:12.476000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:03:12.476000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:03:12.476000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:03:12.476000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:03:12.476000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:03:12.476000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:03:12.476000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:03:12.476000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:03:12.476000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:03:12.476000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:03:12.476000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:03:12.476000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:03:12.477298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:03:12.477298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:03:12.477298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:03:12.477298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:03:12.477298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:03:12.477298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:03:12.477298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:03:12.477298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:03:12.477298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:03:12.477298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:03:12.477298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:03:12.477298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:03:12.477298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:03:12.477298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:03:12.477298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 640/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 23 07:03:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:03:12.485257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5bf5f4-f210-4762-9983-b32fdae0b49d None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:12.505503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:12.588435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5bf5f4-f210-4762-9983-b32fdae0b49d None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:12.627340 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642eb244-911e-4149-888b-7e836257090a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:12.627983 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642eb244-911e-4149-888b-7e836257090a None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:12.628261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642eb244-911e-4149-888b-7e836257090a None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:12.628537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642eb244-911e-4149-888b-7e836257090a None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:12.629173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642eb244-911e-4149-888b-7e836257090a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:03:12.629479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642eb244-911e-4149-888b-7e836257090a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:12.630859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642eb244-911e-4149-888b-7e836257090a None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:12.636578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 659/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Aug 23 07:03:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:03:12.644671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf469d-eccc-4de0-aea4-57f5fe39b11b None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:12.679963 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:12.755343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf469d-eccc-4de0-aea4-57f5fe39b11b None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:12.792826 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9c299-885f-40be-a3ce-f703e7a52896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:12.793422 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9c299-885f-40be-a3ce-f703e7a52896 None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:12.793700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9c299-885f-40be-a3ce-f703e7a52896 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:12.793970 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9c299-885f-40be-a3ce-f703e7a52896 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:12.794669 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9c299-885f-40be-a3ce-f703e7a52896 None admin] RBAC: Adding query filter params (interface=public, service_id=4902f47d70614b0f96e5119992a76156, region_id=RegionOne) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:03:12.794972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9c299-885f-40be-a3ce-f703e7a52896 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:12.796373 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9c299-885f-40be-a3ce-f703e7a52896 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:12.802697 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 641/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Aug 23 07:03:12 2022] GET /identity/v3/endpoints?service_id=4902f47d70614b0f96e5119992a76156&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:03:12.809350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642eb244-911e-4149-888b-7e836257090a None admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:12.909355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:12.931306 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642eb244-911e-4149-888b-7e836257090a None admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:12.970392 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7972d11-912e-4ce5-a93c-a606cab7dd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:12.970987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7972d11-912e-4ce5-a93c-a606cab7dd5c None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:12.971268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7972d11-912e-4ce5-a93c-a606cab7dd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:12.971574 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7972d11-912e-4ce5-a93c-a606cab7dd5c None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:12.972156 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7972d11-912e-4ce5-a93c-a606cab7dd5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:12.973647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7972d11-912e-4ce5-a93c-a606cab7dd5c None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:12.980522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 660/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Aug 23 07:03:12 2022] GET /identity/v3/services => generated 2191 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:03:14.881631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6208032-2864-4832-9f47-7fe2fbc54e73 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:14.882519 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6208032-2864-4832-9f47-7fe2fbc54e73 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:14.882519 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6208032-2864-4832-9f47-7fe2fbc54e73 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:14.884690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 642/1302] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:03:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:03:14.890110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0d655-7e4b-4d69-8889-9553d98601d1 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:14.890525 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0d655-7e4b-4d69-8889-9553d98601d1 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:14.890634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0d655-7e4b-4d69-8889-9553d98601d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:14.910648 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-43b0d655-7e4b-4d69-8889-9553d98601d1 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:03:14.911786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-43b0d655-7e4b-4d69-8889-9553d98601d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:03:14.991199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 661/1303] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:03:14 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:03:15.261329 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6208032-2864-4832-9f47-7fe2fbc54e73 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:15.367365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:15.375545 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6208032-2864-4832-9f47-7fe2fbc54e73 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:15.414238 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd07ac-8203-4fff-9f98-b5afbc4e30ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:15.414855 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd07ac-8203-4fff-9f98-b5afbc4e30ed None admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:15.415130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd07ac-8203-4fff-9f98-b5afbc4e30ed None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:15.415443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd07ac-8203-4fff-9f98-b5afbc4e30ed None admin] PATH_INFO: `/v3/services/placement` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:15.416010 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd07ac-8203-4fff-9f98-b5afbc4e30ed None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:15.417388 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd07ac-8203-4fff-9f98-b5afbc4e30ed None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:15.426712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application [None req-71fd07ac-8203-4fff-9f98-b5afbc4e30ed None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:03:15.426712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:03:15.426712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:03:15.426712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:03:15.426712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:03:15.426712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:03:15.426712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:03:15.426712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:03:15.426712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:03:15.426712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:03:15.426712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:03:15.426712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:03:15.426712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:03:15.426712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:03:15.426712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:03:15.426712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:03:15.426712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:03:15.427893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:03:15.427893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:03:15.427893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:03:15.427893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:03:15.427893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:03:15.427893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:03:15.427893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:03:15.427893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:03:15.427893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:03:15.427893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:03:15.427893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:03:15.427893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:03:15.427893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:03:15.427893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:03:15.427893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:03:15.427893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:03:15.428935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:03:15.428935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application with Lock( Aug 23 07:03:15.428935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:03:15.428935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:03:15.428935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:03:15.428935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:03:15.428935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:03:15.428935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:03:15.428935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:03:15.428935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:03:15.428935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:03:15.428935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:03:15.428935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:03:15.428935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: ERROR keystone.server.flask.application Aug 23 07:03:15.428935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 643/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Aug 23 07:03:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:03:15.434861 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0d655-7e4b-4d69-8889-9553d98601d1 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:15.530235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:15.558648 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0d655-7e4b-4d69-8889-9553d98601d1 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:15.598850 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8be5b-486d-4dfa-9397-33ae43f9379e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:15.599463 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8be5b-486d-4dfa-9397-33ae43f9379e None admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:15.599757 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8be5b-486d-4dfa-9397-33ae43f9379e None admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:15.600029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8be5b-486d-4dfa-9397-33ae43f9379e None admin] PATH_INFO: `/v3/services` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:15.600658 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8be5b-486d-4dfa-9397-33ae43f9379e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:03:15.600960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8be5b-486d-4dfa-9397-33ae43f9379e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:15.602344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8be5b-486d-4dfa-9397-33ae43f9379e None admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:15.608139 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 662/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Aug 23 07:03:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:03:15.617319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd07ac-8203-4fff-9f98-b5afbc4e30ed None admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:03:15.733932 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd07ac-8203-4fff-9f98-b5afbc4e30ed None admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:03:15.778976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27e677-71df-4225-b407-97f5ee691898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:03:15.779969 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27e677-71df-4225-b407-97f5ee691898 None admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:03:15.780432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27e677-71df-4225-b407-97f5ee691898 None admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:03:15.780700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27e677-71df-4225-b407-97f5ee691898 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:03:15.781701 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27e677-71df-4225-b407-97f5ee691898 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:03:15.783294 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27e677-71df-4225-b407-97f5ee691898 None admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:03:15.810535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 644/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:03:15 2022] POST /identity/v3/endpoints => generated 331 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:03:25.604461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=30 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:03:25.803217 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:20.336894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a4921-7955-4413-8f89-f0910d45d480 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:20.338357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a4921-7955-4413-8f89-f0910d45d480 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:20.338357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a4921-7955-4413-8f89-f0910d45d480 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:20.343818 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 663/1307] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:04:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:04:20.345315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc9df8-8164-4baa-86ca-9142040eff64 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:20.346200 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc9df8-8164-4baa-86ca-9142040eff64 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:20.346909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc9df8-8164-4baa-86ca-9142040eff64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:20.514653 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-45dc9df8-8164-4baa-86ca-9142040eff64 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:04:20.515916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-45dc9df8-8164-4baa-86ca-9142040eff64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:20.695589 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 645/1308] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:04:21.007055 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33eeb4f-7402-458d-8c54-214a7368cc7d None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:21.007438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33eeb4f-7402-458d-8c54-214a7368cc7d None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:21.007666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33eeb4f-7402-458d-8c54-214a7368cc7d None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:21.009491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 664/1309] 10.222.0.17 () {54 vars in 965 bytes} [Tue Aug 23 07:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:04:21.016939 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057eac71-59d5-45b8-a67a-a85fe49b5fd4 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:21.017270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057eac71-59d5-45b8-a67a-a85fe49b5fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:21.017483 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057eac71-59d5-45b8-a67a-a85fe49b5fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:21.109790 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-057eac71-59d5-45b8-a67a-a85fe49b5fd4 None None] MFA Rules not processed for user `938364c02d8d4dd2ae832c5129be3cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:04:21.111197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-057eac71-59d5-45b8-a67a-a85fe49b5fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:21.135514 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:21.203026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 646/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Tue Aug 23 07:04:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:04:21.213117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33eeb4f-7402-458d-8c54-214a7368cc7d None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:04:21.214914 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-f33eeb4f-7402-458d-8c54-214a7368cc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:21.344485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33eeb4f-7402-458d-8c54-214a7368cc7d None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:04:21.424523 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59c1d0-72aa-45fb-b4e0-14256b4ccde3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:04:21.425146 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59c1d0-72aa-45fb-b4e0-14256b4ccde3 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:21.425617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59c1d0-72aa-45fb-b4e0-14256b4ccde3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:21.426033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59c1d0-72aa-45fb-b4e0-14256b4ccde3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:21.427805 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-1a59c1d0-72aa-45fb-b4e0-14256b4ccde3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:21.488501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:21.546839 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59c1d0-72aa-45fb-b4e0-14256b4ccde3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:04:21.548720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59c1d0-72aa-45fb-b4e0-14256b4ccde3 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:04:21.570235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 665/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:04:23.827365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cee5bc-59da-41f1-9f0e-0beb17e571a6 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:23.828011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cee5bc-59da-41f1-9f0e-0beb17e571a6 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:23.828011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cee5bc-59da-41f1-9f0e-0beb17e571a6 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:23.829306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 647/1312] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:04:23.834787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc38de7-57b8-4a95-84f2-131a0aaec0e1 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:23.835242 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc38de7-57b8-4a95-84f2-131a0aaec0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:23.835345 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc38de7-57b8-4a95-84f2-131a0aaec0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:23.921792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-bdc38de7-57b8-4a95-84f2-131a0aaec0e1 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:04:23.922939 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-bdc38de7-57b8-4a95-84f2-131a0aaec0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:24.026814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 666/1313] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:04:24.306260 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cee5bc-59da-41f1-9f0e-0beb17e571a6 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:04:24.308758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-88cee5bc-59da-41f1-9f0e-0beb17e571a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:24.423863 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cee5bc-59da-41f1-9f0e-0beb17e571a6 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:04:24.496474 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:24.500084 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610af4e-ca49-44c7-ba31-f6be314254c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:04:24.501010 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610af4e-ca49-44c7-ba31-f6be314254c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:24.501373 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610af4e-ca49-44c7-ba31-f6be314254c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:24.501715 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610af4e-ca49-44c7-ba31-f6be314254c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:24.505395 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2610af4e-ca49-44c7-ba31-f6be314254c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:24.632988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610af4e-ca49-44c7-ba31-f6be314254c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:04:24.634450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610af4e-ca49-44c7-ba31-f6be314254c2 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:04:24.654569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 648/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:04:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:04:26.934584 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07ffb4-d21d-4aa7-bad0-b6d492406a8f None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:26.935477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07ffb4-d21d-4aa7-bad0-b6d492406a8f None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:26.935477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07ffb4-d21d-4aa7-bad0-b6d492406a8f None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:26.936865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 667/1315] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:04:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:04:26.946121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff1726-67cc-4be2-bc88-300a86ced0b9 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:26.946827 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff1726-67cc-4be2-bc88-300a86ced0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:26.946988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff1726-67cc-4be2-bc88-300a86ced0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:27.025713 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-11ff1726-67cc-4be2-bc88-300a86ced0b9 None None] MFA Rules not processed for user `1eb74ccd914b40dbac0acc5a1d975c76`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:04:27.026926 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-11ff1726-67cc-4be2-bc88-300a86ced0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:27.090456 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:27.140015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 649/1316] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:04:26 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:04:27.442686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07ffb4-d21d-4aa7-bad0-b6d492406a8f None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:04:27.445236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4a07ffb4-d21d-4aa7-bad0-b6d492406a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:27.458463 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:27.563258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07ffb4-d21d-4aa7-bad0-b6d492406a8f None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:04:27.645314 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7b6d2-181b-48cd-97e7-a06b59b46493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:04:27.645919 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7b6d2-181b-48cd-97e7-a06b59b46493 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:27.646193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7b6d2-181b-48cd-97e7-a06b59b46493 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:27.646464 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7b6d2-181b-48cd-97e7-a06b59b46493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:27.648082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8bb7b6d2-181b-48cd-97e7-a06b59b46493 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:27.715403 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:27.774281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7b6d2-181b-48cd-97e7-a06b59b46493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:04:27.775767 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7b6d2-181b-48cd-97e7-a06b59b46493 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:04:27.802186 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 668/1317] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:04:27 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:04:30.268736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adbd1d-22fc-4074-8627-cecba9e25fde None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:30.269564 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adbd1d-22fc-4074-8627-cecba9e25fde None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:30.269564 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adbd1d-22fc-4074-8627-cecba9e25fde None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:30.271736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 650/1318] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:04:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:04:30.277244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb94207-e8b1-44d4-a070-28a8b1ed128b None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:30.277666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb94207-e8b1-44d4-a070-28a8b1ed128b None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:30.277775 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb94207-e8b1-44d4-a070-28a8b1ed128b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:30.347926 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-8fb94207-e8b1-44d4-a070-28a8b1ed128b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:04:30.349176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8fb94207-e8b1-44d4-a070-28a8b1ed128b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:30.452951 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 669/1319] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:04:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:04:30.746268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03adbd1d-22fc-4074-8627-cecba9e25fde None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:04:30.748734 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-03adbd1d-22fc-4074-8627-cecba9e25fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:30.865223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03adbd1d-22fc-4074-8627-cecba9e25fde None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:04:30.945974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce45ef4-9f39-43f6-84c1-709d83657627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:04:30.946491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce45ef4-9f39-43f6-84c1-709d83657627 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:30.946776 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce45ef4-9f39-43f6-84c1-709d83657627 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:30.947044 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce45ef4-9f39-43f6-84c1-709d83657627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:30.948694 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2ce45ef4-9f39-43f6-84c1-709d83657627 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:30.957180 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:31.064456 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce45ef4-9f39-43f6-84c1-709d83657627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:04:31.065910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce45ef4-9f39-43f6-84c1-709d83657627 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:04:31.086207 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 651/1320] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:04:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:04:33.711077 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a16843-928d-4016-bdac-c98f47fecca7 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:33.711765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a16843-928d-4016-bdac-c98f47fecca7 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:33.711765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a16843-928d-4016-bdac-c98f47fecca7 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:33.713021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 670/1321] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:04:33.718530 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7a3fd-a7c4-4c73-b07b-c42b0f18900b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:33.718951 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7a3fd-a7c4-4c73-b07b-c42b0f18900b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:33.719166 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7a3fd-a7c4-4c73-b07b-c42b0f18900b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:33.788526 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-04b7a3fd-a7c4-4c73-b07b-c42b0f18900b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:04:33.789764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-04b7a3fd-a7c4-4c73-b07b-c42b0f18900b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:33.849385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:33.889509 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 652/1322] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:04:34.170796 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a16843-928d-4016-bdac-c98f47fecca7 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:04:34.172443 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b6a16843-928d-4016-bdac-c98f47fecca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:34.193110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:34.280828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a16843-928d-4016-bdac-c98f47fecca7 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:04:34.363309 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a7093-f442-4db5-85d9-1c6b792b97d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:04:34.363996 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a7093-f442-4db5-85d9-1c6b792b97d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:34.364250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a7093-f442-4db5-85d9-1c6b792b97d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:34.364522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a7093-f442-4db5-85d9-1c6b792b97d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:34.366146 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b75a7093-f442-4db5-85d9-1c6b792b97d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:34.439823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:34.480093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a7093-f442-4db5-85d9-1c6b792b97d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:04:34.481547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a7093-f442-4db5-85d9-1c6b792b97d0 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:04:34.501652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 671/1323] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:04:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:04:36.455266 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b0875-8bc1-455a-bbdc-ec96ba7ad997 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:36.455841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b0875-8bc1-455a-bbdc-ec96ba7ad997 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:36.455841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b0875-8bc1-455a-bbdc-ec96ba7ad997 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:36.457222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 653/1324] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:04:36.462697 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169dc96-15e9-489c-9f6f-96c7cb8a476b None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:36.463110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169dc96-15e9-489c-9f6f-96c7cb8a476b None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:36.463255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169dc96-15e9-489c-9f6f-96c7cb8a476b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:36.530795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-1169dc96-15e9-489c-9f6f-96c7cb8a476b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:04:36.532052 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-1169dc96-15e9-489c-9f6f-96c7cb8a476b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:36.632058 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 672/1325] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:04:36.927027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28df030-58bb-4954-90ea-1ff7ef585dde None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:36.927344 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28df030-58bb-4954-90ea-1ff7ef585dde None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:36.927640 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28df030-58bb-4954-90ea-1ff7ef585dde None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:36.929026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 654/1326] 10.222.0.17 () {54 vars in 965 bytes} [Tue Aug 23 07:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:04:36.935775 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef8c4c-42ff-4486-ab2a-0604f5eea181 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:36.936062 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef8c4c-42ff-4486-ab2a-0604f5eea181 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:36.936342 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef8c4c-42ff-4486-ab2a-0604f5eea181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:36.981558 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:37.028391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-bbef8c4c-42ff-4486-ab2a-0604f5eea181 None None] MFA Rules not processed for user `938364c02d8d4dd2ae832c5129be3cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:04:37.029913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-bbef8c4c-42ff-4486-ab2a-0604f5eea181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:37.108578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 673/1327] 10.222.0.17 () {58 vars in 1062 bytes} [Tue Aug 23 07:04:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:04:37.117610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28df030-58bb-4954-90ea-1ff7ef585dde None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:04:37.119144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c28df030-58bb-4954-90ea-1ff7ef585dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:37.226130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28df030-58bb-4954-90ea-1ff7ef585dde None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:04:37.305834 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b7cc1-40e9-4130-815b-06c3676b95ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:04:37.306352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b7cc1-40e9-4130-815b-06c3676b95ac service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:37.306686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b7cc1-40e9-4130-815b-06c3676b95ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:37.306880 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b7cc1-40e9-4130-815b-06c3676b95ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:37.308574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-bf3b7cc1-40e9-4130-815b-06c3676b95ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:37.327105 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:37.421252 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b7cc1-40e9-4130-815b-06c3676b95ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:04:37.422718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b7cc1-40e9-4130-815b-06c3676b95ac service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:04:37.441437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 655/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:04:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:04:39.594631 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c8a8e-309f-4707-8b7c-128c03a2a833 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:39.595156 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c8a8e-309f-4707-8b7c-128c03a2a833 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:39.595156 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c8a8e-309f-4707-8b7c-128c03a2a833 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:39.596682 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 674/1329] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:04:39.602540 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0036ec3-8c0c-4557-b9c1-b699760f2f2e None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:39.603096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0036ec3-8c0c-4557-b9c1-b699760f2f2e None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:39.603798 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0036ec3-8c0c-4557-b9c1-b699760f2f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:39.676743 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-a0036ec3-8c0c-4557-b9c1-b699760f2f2e None None] MFA Rules not processed for user `1eb74ccd914b40dbac0acc5a1d975c76`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:04:39.677982 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-a0036ec3-8c0c-4557-b9c1-b699760f2f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:39.749738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:39.782897 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 656/1330] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:04:39 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:04:40.063987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c8a8e-309f-4707-8b7c-128c03a2a833 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:04:40.065598 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-149c8a8e-309f-4707-8b7c-128c03a2a833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:40.188050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c8a8e-309f-4707-8b7c-128c03a2a833 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:04:40.212006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:40.273058 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1ca94-b3ae-4b7e-bca5-413494997402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:04:40.273681 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1ca94-b3ae-4b7e-bca5-413494997402 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:40.273956 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1ca94-b3ae-4b7e-bca5-413494997402 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:40.274227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1ca94-b3ae-4b7e-bca5-413494997402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:40.275936 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6ea1ca94-b3ae-4b7e-bca5-413494997402 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:40.397827 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1ca94-b3ae-4b7e-bca5-413494997402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:04:40.399278 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1ca94-b3ae-4b7e-bca5-413494997402 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:04:40.417778 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 675/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:04:40 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:04:42.749676 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bcb4d-f332-41cd-9118-720577741a1f None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:42.750231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bcb4d-f332-41cd-9118-720577741a1f None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:42.750231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bcb4d-f332-41cd-9118-720577741a1f None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:42.751632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 657/1332] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:04:42.758059 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d446618-8565-48ed-a478-f15b3a0c3842 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:42.758498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d446618-8565-48ed-a478-f15b3a0c3842 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:42.758671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d446618-8565-48ed-a478-f15b3a0c3842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:42.828673 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:42.833921 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-8d446618-8565-48ed-a478-f15b3a0c3842 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:04:42.836349 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8d446618-8565-48ed-a478-f15b3a0c3842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:42.937550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 676/1333] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:04:43.223343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bcb4d-f332-41cd-9118-720577741a1f None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:04:43.224937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3e4bcb4d-f332-41cd-9118-720577741a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:43.341423 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bcb4d-f332-41cd-9118-720577741a1f None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:04:43.424303 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78708e-060e-49be-a296-dee02018d2b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:04:43.424919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78708e-060e-49be-a296-dee02018d2b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:43.425198 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78708e-060e-49be-a296-dee02018d2b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:43.425470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78708e-060e-49be-a296-dee02018d2b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:43.427150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-6d78708e-060e-49be-a296-dee02018d2b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:43.463703 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:43.552674 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78708e-060e-49be-a296-dee02018d2b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:04:43.554054 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78708e-060e-49be-a296-dee02018d2b0 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:04:43.572725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 658/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:04:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:04:45.500444 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16981add-2a60-4d2d-a6b9-41fb2a370216 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:45.501316 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16981add-2a60-4d2d-a6b9-41fb2a370216 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:45.501316 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16981add-2a60-4d2d-a6b9-41fb2a370216 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:45.503634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 677/1335] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:04:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:04:45.509883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a676ff-5251-419d-91d2-1608fefe58fa None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:45.510643 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a676ff-5251-419d-91d2-1608fefe58fa None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:45.510776 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a676ff-5251-419d-91d2-1608fefe58fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:45.581911 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-75a676ff-5251-419d-91d2-1608fefe58fa None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:04:45.583114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-75a676ff-5251-419d-91d2-1608fefe58fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:45.683230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:45.718111 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 659/1336] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:04:46.011115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16981add-2a60-4d2d-a6b9-41fb2a370216 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:04:46.012814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-16981add-2a60-4d2d-a6b9-41fb2a370216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:46.136863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16981add-2a60-4d2d-a6b9-41fb2a370216 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:04:46.172389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:46.220098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f79b1-33c0-4ed7-a890-69eac3e81330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:04:46.220703 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f79b1-33c0-4ed7-a890-69eac3e81330 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:46.220976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f79b1-33c0-4ed7-a890-69eac3e81330 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:46.221239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f79b1-33c0-4ed7-a890-69eac3e81330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:46.222818 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-865f79b1-33c0-4ed7-a890-69eac3e81330 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:46.345346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f79b1-33c0-4ed7-a890-69eac3e81330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:04:46.348692 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f79b1-33c0-4ed7-a890-69eac3e81330 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:04:46.366306 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 678/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:04:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:04:48.600732 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c185da-6817-4ce7-a227-151b4232ac45 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:48.601596 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c185da-6817-4ce7-a227-151b4232ac45 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:48.601596 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c185da-6817-4ce7-a227-151b4232ac45 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:48.603809 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 660/1338] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:04:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:04:48.609951 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7c3b5-7754-4226-a8f3-ad5a5a5a2c6a None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:48.610613 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7c3b5-7754-4226-a8f3-ad5a5a5a2c6a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:48.610704 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7c3b5-7754-4226-a8f3-ad5a5a5a2c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:48.702134 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-e5d7c3b5-7754-4226-a8f3-ad5a5a5a2c6a None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:04:48.703364 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e5d7c3b5-7754-4226-a8f3-ad5a5a5a2c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:48.712341 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:48.817309 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 679/1339] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:04:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:04:49.100480 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c185da-6817-4ce7-a227-151b4232ac45 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:04:49.102907 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-a3c185da-6817-4ce7-a227-151b4232ac45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:49.216439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c185da-6817-4ce7-a227-151b4232ac45 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:04:49.291572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90414a-cf84-4712-b438-a0ddfe59d43a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:04:49.292186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90414a-cf84-4712-b438-a0ddfe59d43a service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:49.292459 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90414a-cf84-4712-b438-a0ddfe59d43a service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:49.292729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90414a-cf84-4712-b438-a0ddfe59d43a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:49.294405 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-4f90414a-cf84-4712-b438-a0ddfe59d43a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:49.331364 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:49.411678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f90414a-cf84-4712-b438-a0ddfe59d43a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:04:49.413138 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f90414a-cf84-4712-b438-a0ddfe59d43a service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:04:49.431063 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 661/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:04:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:04:53.309439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fb1e4-7833-41aa-bdf9-a7740ca7faf2 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:53.310090 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fb1e4-7833-41aa-bdf9-a7740ca7faf2 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:53.310090 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fb1e4-7833-41aa-bdf9-a7740ca7faf2 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:53.311615 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 680/1341] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:04:53.317214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522925f1-6b45-4809-b7d9-2afc28d795ec None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:53.317630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522925f1-6b45-4809-b7d9-2afc28d795ec None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:53.317839 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522925f1-6b45-4809-b7d9-2afc28d795ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:53.387471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-522925f1-6b45-4809-b7d9-2afc28d795ec None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:04:53.388830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-522925f1-6b45-4809-b7d9-2afc28d795ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:53.480291 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:53.493085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 662/1342] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:04:53.772382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fb1e4-7833-41aa-bdf9-a7740ca7faf2 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:04:53.773926 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e53fb1e4-7833-41aa-bdf9-a7740ca7faf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:53.882165 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fb1e4-7833-41aa-bdf9-a7740ca7faf2 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:04:53.927238 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:53.963901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667ae57-5a15-40a9-b8ec-caf15a408f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:04:53.964497 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667ae57-5a15-40a9-b8ec-caf15a408f84 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:53.964768 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667ae57-5a15-40a9-b8ec-caf15a408f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:53.965029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667ae57-5a15-40a9-b8ec-caf15a408f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:53.966888 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9667ae57-5a15-40a9-b8ec-caf15a408f84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:54.082892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667ae57-5a15-40a9-b8ec-caf15a408f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:04:54.084313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667ae57-5a15-40a9-b8ec-caf15a408f84 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:04:54.103021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 681/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:04:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:04:56.822818 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584cfeb-077d-4bb9-afd8-1283d5fe7255 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:56.823407 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584cfeb-077d-4bb9-afd8-1283d5fe7255 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:56.823407 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584cfeb-077d-4bb9-afd8-1283d5fe7255 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:56.825122 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 663/1344] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:04:56.831214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00767b9-4ece-4dd6-a176-8b0d246631d0 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:56.831709 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00767b9-4ece-4dd6-a176-8b0d246631d0 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:56.831813 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00767b9-4ece-4dd6-a176-8b0d246631d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:56.904902 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-d00767b9-4ece-4dd6-a176-8b0d246631d0 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:04:56.906119 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d00767b9-4ece-4dd6-a176-8b0d246631d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:56.925407 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:57.007785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 682/1345] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:04:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:04:57.294316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584cfeb-077d-4bb9-afd8-1283d5fe7255 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:04:57.295981 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8584cfeb-077d-4bb9-afd8-1283d5fe7255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:57.411053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584cfeb-077d-4bb9-afd8-1283d5fe7255 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:04:57.493987 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b08d67-9a68-4388-b17e-26a98bca514d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:04:57.494514 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b08d67-9a68-4388-b17e-26a98bca514d service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:04:57.494786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b08d67-9a68-4388-b17e-26a98bca514d service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:04:57.495056 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b08d67-9a68-4388-b17e-26a98bca514d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:04:57.496730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-14b08d67-9a68-4388-b17e-26a98bca514d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:04:57.543170 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:04:57.617830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b08d67-9a68-4388-b17e-26a98bca514d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:04:57.619321 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b08d67-9a68-4388-b17e-26a98bca514d service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:04:57.640807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 664/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:04:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:05:01.626065 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb10a6-aa40-4222-a549-9c09ac913fbd None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:01.626632 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb10a6-aa40-4222-a549-9c09ac913fbd None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:01.626632 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb10a6-aa40-4222-a549-9c09ac913fbd None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:01.628055 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 683/1347] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:05:01.633931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3dfd5-c695-4258-94dc-1548d0a769cd None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:01.634447 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3dfd5-c695-4258-94dc-1548d0a769cd None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:01.634447 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3dfd5-c695-4258-94dc-1548d0a769cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:01.707305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-f7b3dfd5-c695-4258-94dc-1548d0a769cd None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:01.708566 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f7b3dfd5-c695-4258-94dc-1548d0a769cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:01.808784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:01.814858 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 665/1348] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:02.099747 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb10a6-aa40-4222-a549-9c09ac913fbd None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:05:02.101293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-defb10a6-aa40-4222-a549-9c09ac913fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:02.220805 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb10a6-aa40-4222-a549-9c09ac913fbd None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:05:02.272102 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:02.298994 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c8dfa-e629-4a2b-b7c0-5ab052dea021 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:05:02.299647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c8dfa-e629-4a2b-b7c0-5ab052dea021 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:02.299933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c8dfa-e629-4a2b-b7c0-5ab052dea021 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:02.300211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c8dfa-e629-4a2b-b7c0-5ab052dea021 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:02.301991 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-801c8dfa-e629-4a2b-b7c0-5ab052dea021 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:02.417051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c8dfa-e629-4a2b-b7c0-5ab052dea021 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:05:02.419171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c8dfa-e629-4a2b-b7c0-5ab052dea021 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:05:02.442985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 684/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:05:07.389359 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c80ad3-080d-4cfb-9716-d1bb255ef728 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:07.389934 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c80ad3-080d-4cfb-9716-d1bb255ef728 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:07.390325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c80ad3-080d-4cfb-9716-d1bb255ef728 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:07.392104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 666/1350] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 23 07:05:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:07.397979 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491a523-44f7-4138-b794-8826c80c7d91 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:07.398741 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491a523-44f7-4138-b794-8826c80c7d91 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:07.399159 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491a523-44f7-4138-b794-8826c80c7d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:07.510410 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-6491a523-44f7-4138-b794-8826c80c7d91 None None] MFA Rules not processed for user `a9b0f54c988e43bda474f978a69fa4fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:07.514325 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6491a523-44f7-4138-b794-8826c80c7d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:07.544738 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:07.665830 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 685/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 23 07:05:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:11.953111 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117ee13-deef-4c1d-96e8-81fc0befdbe1 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:11.953645 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117ee13-deef-4c1d-96e8-81fc0befdbe1 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:11.953645 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117ee13-deef-4c1d-96e8-81fc0befdbe1 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:11.955100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 667/1352] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 23 07:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:05:11.960061 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ec789-d617-4e2b-9b5d-8493c768e7c0 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:11.960544 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ec789-d617-4e2b-9b5d-8493c768e7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:11.960806 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ec789-d617-4e2b-9b5d-8493c768e7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:12.046011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-db6ec789-d617-4e2b-9b5d-8493c768e7c0 None None] MFA Rules not processed for user `a9b0f54c988e43bda474f978a69fa4fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:12.047226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-db6ec789-d617-4e2b-9b5d-8493c768e7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:12.125472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:12.163058 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 686/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 23 07:05:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:13.499329 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3d33b-5d54-4ac7-835a-753542fe3638 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:13.499899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3d33b-5d54-4ac7-835a-753542fe3638 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:13.499999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3d33b-5d54-4ac7-835a-753542fe3638 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:13.501312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 668/1354] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 23 07:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:13.506693 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06c175-0485-4634-b01d-62861ca1926e None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:13.507107 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06c175-0485-4634-b01d-62861ca1926e None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:13.507368 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06c175-0485-4634-b01d-62861ca1926e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:13.588400 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-aa06c175-0485-4634-b01d-62861ca1926e None None] MFA Rules not processed for user `a9b0f54c988e43bda474f978a69fa4fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:13.589666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-aa06c175-0485-4634-b01d-62861ca1926e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:13.707067 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 687/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 23 07:05:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:16.929978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c77383-92bf-4b66-8de1-10d1ea41b5bf None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:16.930484 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c77383-92bf-4b66-8de1-10d1ea41b5bf None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:16.930484 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c77383-92bf-4b66-8de1-10d1ea41b5bf None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:16.931851 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 669/1356] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:16.937437 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30d9dd-be5b-4e8f-a5c1-0f311f6a6773 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:16.937827 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30d9dd-be5b-4e8f-a5c1-0f311f6a6773 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:16.937989 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30d9dd-be5b-4e8f-a5c1-0f311f6a6773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:16.991319 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:17.025435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-3f30d9dd-be5b-4e8f-a5c1-0f311f6a6773 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:17.026779 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3f30d9dd-be5b-4e8f-a5c1-0f311f6a6773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:17.186880 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 688/1357] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:17.501444 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9016b8-9f42-40b0-a771-672f2c6bf675 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:17.502163 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9016b8-9f42-40b0-a771-672f2c6bf675 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:17.502627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9016b8-9f42-40b0-a771-672f2c6bf675 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:17.504887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 670/1358] 10.222.0.17 () {54 vars in 962 bytes} [Tue Aug 23 07:05:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:17.514863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515886f9-92d2-41bf-9bbb-bdfcb10c6e7e None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:17.516355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515886f9-92d2-41bf-9bbb-bdfcb10c6e7e None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:17.517058 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515886f9-92d2-41bf-9bbb-bdfcb10c6e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:17.599207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-515886f9-92d2-41bf-9bbb-bdfcb10c6e7e None None] MFA Rules not processed for user `a8d2cd93042c4583b63afe629d47fce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:17.600520 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-515886f9-92d2-41bf-9bbb-bdfcb10c6e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:17.658302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:17.732071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 689/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Tue Aug 23 07:05:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:17.740286 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9016b8-9f42-40b0-a771-672f2c6bf675 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:05:17.742161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9e9016b8-9f42-40b0-a771-672f2c6bf675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:17.906156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9016b8-9f42-40b0-a771-672f2c6bf675 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:05:17.987743 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:18.032823 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6904f-7ff0-4264-83bf-f79d74311a81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:05:18.033583 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6904f-7ff0-4264-83bf-f79d74311a81 service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:18.033965 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6904f-7ff0-4264-83bf-f79d74311a81 service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:18.034338 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6904f-7ff0-4264-83bf-f79d74311a81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:18.036094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-cfe6904f-7ff0-4264-83bf-f79d74311a81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:18.134666 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6904f-7ff0-4264-83bf-f79d74311a81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:05:18.136519 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6904f-7ff0-4264-83bf-f79d74311a81 service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:05:18.158100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 671/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:05:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:05:20.265528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e07faf-9c88-4096-bd8f-51bf5fe7eecc None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:20.266082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e07faf-9c88-4096-bd8f-51bf5fe7eecc None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:20.266082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e07faf-9c88-4096-bd8f-51bf5fe7eecc None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:20.267602 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 690/1361] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:20.272789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b592d-d471-40e9-9c17-d2fe8f45f47b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:20.273230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b592d-d471-40e9-9c17-d2fe8f45f47b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:20.273325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b592d-d471-40e9-9c17-d2fe8f45f47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:20.346653 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-3b3b592d-d471-40e9-9c17-d2fe8f45f47b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:20.348052 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3b3b592d-d471-40e9-9c17-d2fe8f45f47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:20.388885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:20.463459 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 672/1362] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:20.756558 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e07faf-9c88-4096-bd8f-51bf5fe7eecc None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:05:20.758238 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-f6e07faf-9c88-4096-bd8f-51bf5fe7eecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:20.915993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e07faf-9c88-4096-bd8f-51bf5fe7eecc None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:05:20.926697 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:21.044501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089012f-064f-490a-a156-c9a48530f69c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:05:21.045124 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089012f-064f-490a-a156-c9a48530f69c service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:21.045406 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089012f-064f-490a-a156-c9a48530f69c service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:21.045679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089012f-064f-490a-a156-c9a48530f69c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:21.048016 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3089012f-064f-490a-a156-c9a48530f69c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:21.150452 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089012f-064f-490a-a156-c9a48530f69c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:05:21.152149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089012f-064f-490a-a156-c9a48530f69c service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:05:21.172771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 691/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:05:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:05:23.386054 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e60a3a-5213-44ce-ae38-4325daae200f None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:23.386625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e60a3a-5213-44ce-ae38-4325daae200f None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:23.386907 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e60a3a-5213-44ce-ae38-4325daae200f None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:23.388509 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 673/1364] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:05:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:23.393932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5deb4-958d-4651-92a4-aae83e33af88 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:23.394771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5deb4-958d-4651-92a4-aae83e33af88 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:23.395273 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5deb4-958d-4651-92a4-aae83e33af88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:23.412607 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:23.471335 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-fdf5deb4-958d-4651-92a4-aae83e33af88 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:23.472769 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-fdf5deb4-958d-4651-92a4-aae83e33af88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:23.587275 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 692/1365] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:23.860301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72a64f-2665-4ea3-9417-d906466ae8c1 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:23.860587 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72a64f-2665-4ea3-9417-d906466ae8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:23.860865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72a64f-2665-4ea3-9417-d906466ae8c1 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:23.862339 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 674/1366] 10.222.0.17 () {54 vars in 962 bytes} [Tue Aug 23 07:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:23.867776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d540d9-777b-4ef9-9bf9-8c4df65e4710 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:23.868249 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d540d9-777b-4ef9-9bf9-8c4df65e4710 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:23.868535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d540d9-777b-4ef9-9bf9-8c4df65e4710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:23.961824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-84d540d9-777b-4ef9-9bf9-8c4df65e4710 None None] MFA Rules not processed for user `a8d2cd93042c4583b63afe629d47fce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:23.963015 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-84d540d9-777b-4ef9-9bf9-8c4df65e4710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:24.003102 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:24.096181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 693/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Tue Aug 23 07:05:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:24.104512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72a64f-2665-4ea3-9417-d906466ae8c1 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:05:24.106197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-eb72a64f-2665-4ea3-9417-d906466ae8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:24.266697 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72a64f-2665-4ea3-9417-d906466ae8c1 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:05:24.287741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:24.390304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289c369-3228-4890-a994-eadf612ae456 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:05:24.390978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289c369-3228-4890-a994-eadf612ae456 service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:24.391370 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289c369-3228-4890-a994-eadf612ae456 service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:24.391600 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289c369-3228-4890-a994-eadf612ae456 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:24.393045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b289c369-3228-4890-a994-eadf612ae456 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:24.494714 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289c369-3228-4890-a994-eadf612ae456 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:05:24.496406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289c369-3228-4890-a994-eadf612ae456 service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:05:24.517195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 675/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:05:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:05:25.036538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64ec8d-3ba4-4da5-bd7a-c870ceb165ae None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:25.037172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64ec8d-3ba4-4da5-bd7a-c870ceb165ae None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:25.037172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64ec8d-3ba4-4da5-bd7a-c870ceb165ae None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:25.039218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 694/1369] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:25.044377 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d03041-e049-4820-8183-f113b3750a2f None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:25.044678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d03041-e049-4820-8183-f113b3750a2f None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:25.044944 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d03041-e049-4820-8183-f113b3750a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:25.079700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:25.117347 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-47d03041-e049-4820-8183-f113b3750a2f None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:25.118688 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-47d03041-e049-4820-8183-f113b3750a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:25.223476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 676/1370] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:25.770990 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69891b6d-f351-481a-a531-b7bd4cd3800c None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:25.771601 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69891b6d-f351-481a-a531-b7bd4cd3800c None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:25.772159 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69891b6d-f351-481a-a531-b7bd4cd3800c None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:25.774007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 695/1371] 10.222.0.17 () {54 vars in 964 bytes} [Tue Aug 23 07:05:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:25.779724 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef39239-c7f3-41c5-93e9-9dd804dc3d4c None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:25.780528 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef39239-c7f3-41c5-93e9-9dd804dc3d4c None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:25.780963 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef39239-c7f3-41c5-93e9-9dd804dc3d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:25.859751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-0ef39239-c7f3-41c5-93e9-9dd804dc3d4c None None] MFA Rules not processed for user `a26f09e892a74d9eab97e73cce9d062a`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:25.860947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-0ef39239-c7f3-41c5-93e9-9dd804dc3d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:25.910442 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:25.944368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 677/1372] 10.222.0.17 () {58 vars in 1061 bytes} [Tue Aug 23 07:05:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:25.952453 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69891b6d-f351-481a-a531-b7bd4cd3800c None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:05:25.955269 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-69891b6d-f351-481a-a531-b7bd4cd3800c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:26.062843 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69891b6d-f351-481a-a531-b7bd4cd3800c None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:05:26.105433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:26.141856 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfece36d-9a53-44ce-b691-67320dae1a98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:05:26.142467 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfece36d-9a53-44ce-b691-67320dae1a98 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:26.142869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfece36d-9a53-44ce-b691-67320dae1a98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:26.143144 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfece36d-9a53-44ce-b691-67320dae1a98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:26.144777 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-bfece36d-9a53-44ce-b691-67320dae1a98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:26.283458 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfece36d-9a53-44ce-b691-67320dae1a98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:05:26.285035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfece36d-9a53-44ce-b691-67320dae1a98 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:05:26.305884 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 696/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:05:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:05:26.492817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60355179-504a-4c01-8a41-d5dd837a04d7 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:26.493182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60355179-504a-4c01-8a41-d5dd837a04d7 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:26.493601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60355179-504a-4c01-8a41-d5dd837a04d7 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:26.495749 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 678/1374] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:05:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:26.501105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f26560-bfa4-4d08-9d85-7b8cfc9b21e4 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:26.501399 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f26560-bfa4-4d08-9d85-7b8cfc9b21e4 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:26.501675 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f26560-bfa4-4d08-9d85-7b8cfc9b21e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:26.582369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-b3f26560-bfa4-4d08-9d85-7b8cfc9b21e4 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:26.583663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b3f26560-bfa4-4d08-9d85-7b8cfc9b21e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:26.604434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:26.686802 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 697/1375] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:26.976404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60355179-504a-4c01-8a41-d5dd837a04d7 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:05:26.977977 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-60355179-504a-4c01-8a41-d5dd837a04d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:27.129594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60355179-504a-4c01-8a41-d5dd837a04d7 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:05:27.181949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:27.246200 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b37542-0bb7-47d4-b3c6-ae60c874def6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:05:27.246816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b37542-0bb7-47d4-b3c6-ae60c874def6 service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:27.247087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b37542-0bb7-47d4-b3c6-ae60c874def6 service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:27.247358 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b37542-0bb7-47d4-b3c6-ae60c874def6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:27.248992 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f0b37542-0bb7-47d4-b3c6-ae60c874def6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:27.340458 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b37542-0bb7-47d4-b3c6-ae60c874def6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:05:27.342087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b37542-0bb7-47d4-b3c6-ae60c874def6 service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:05:27.360480 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 679/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:05:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:05:29.298369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e83a0-8359-4dae-b27e-321e6c172839 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:29.298848 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e83a0-8359-4dae-b27e-321e6c172839 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:29.299049 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e83a0-8359-4dae-b27e-321e6c172839 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:29.300374 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 698/1377] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:29.305824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c8032-43f5-42dd-8047-c49070de6d12 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:29.306232 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c8032-43f5-42dd-8047-c49070de6d12 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:29.306367 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c8032-43f5-42dd-8047-c49070de6d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:29.394342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-c36c8032-43f5-42dd-8047-c49070de6d12 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:29.395692 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c36c8032-43f5-42dd-8047-c49070de6d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:29.403507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:29.497873 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 680/1378] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:29.801380 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e83a0-8359-4dae-b27e-321e6c172839 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:05:29.805807 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8e5e83a0-8359-4dae-b27e-321e6c172839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:29.982974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e83a0-8359-4dae-b27e-321e6c172839 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:05:29.986867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:30.104702 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ee79a-579c-4956-b1d8-32080b90aa06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:05:30.105318 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ee79a-579c-4956-b1d8-32080b90aa06 service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:30.105598 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ee79a-579c-4956-b1d8-32080b90aa06 service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:30.105996 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ee79a-579c-4956-b1d8-32080b90aa06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:30.107776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-426ee79a-579c-4956-b1d8-32080b90aa06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:30.201180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ee79a-579c-4956-b1d8-32080b90aa06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:05:30.202802 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ee79a-579c-4956-b1d8-32080b90aa06 service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:05:30.222167 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 699/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:05:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:05:32.145189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29f0fc-d4a3-48bd-8214-71a2dd2ed599 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:32.145815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29f0fc-d4a3-48bd-8214-71a2dd2ed599 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:32.145815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29f0fc-d4a3-48bd-8214-71a2dd2ed599 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:32.147273 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 681/1380] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:32.152523 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55456f31-5aa7-4cdb-a502-6164d7ba973a None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:32.153303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55456f31-5aa7-4cdb-a502-6164d7ba973a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:32.153417 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55456f31-5aa7-4cdb-a502-6164d7ba973a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:32.165771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:32.236365 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-55456f31-5aa7-4cdb-a502-6164d7ba973a None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:32.237604 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-55456f31-5aa7-4cdb-a502-6164d7ba973a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:32.359548 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 700/1381] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:32.632135 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29f0fc-d4a3-48bd-8214-71a2dd2ed599 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:05:32.633952 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8a29f0fc-d4a3-48bd-8214-71a2dd2ed599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:32.806486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:32.820394 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29f0fc-d4a3-48bd-8214-71a2dd2ed599 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:05:32.950662 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132bfa59-3f07-4337-abd8-cff87f03d02d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:05:32.951273 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132bfa59-3f07-4337-abd8-cff87f03d02d service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:32.951584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132bfa59-3f07-4337-abd8-cff87f03d02d service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:32.951856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132bfa59-3f07-4337-abd8-cff87f03d02d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:32.953498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-132bfa59-3f07-4337-abd8-cff87f03d02d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:33.046680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132bfa59-3f07-4337-abd8-cff87f03d02d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:05:33.048332 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132bfa59-3f07-4337-abd8-cff87f03d02d service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:05:33.059399 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:33.069187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 682/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:05:35.085377 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910b0cb-0336-437f-8f1e-fa45427c8895 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:35.086270 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910b0cb-0336-437f-8f1e-fa45427c8895 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:35.086557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910b0cb-0336-437f-8f1e-fa45427c8895 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:35.088057 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 701/1383] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:05:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:35.094094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06326d-8742-4bab-ac4a-48368192feb7 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:35.094636 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06326d-8742-4bab-ac4a-48368192feb7 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:35.094761 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06326d-8742-4bab-ac4a-48368192feb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:35.164941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-bc06326d-8742-4bab-ac4a-48368192feb7 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:35.166172 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-bc06326d-8742-4bab-ac4a-48368192feb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:35.269352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 683/1384] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:35.582530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910b0cb-0336-437f-8f1e-fa45427c8895 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:05:35.584155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9910b0cb-0336-437f-8f1e-fa45427c8895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:35.687184 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:35.755428 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910b0cb-0336-437f-8f1e-fa45427c8895 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:05:35.887139 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a185b4-a1d6-4802-9062-967d07504230 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:05:35.887963 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a185b4-a1d6-4802-9062-967d07504230 service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:35.888315 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a185b4-a1d6-4802-9062-967d07504230 service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:35.888674 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a185b4-a1d6-4802-9062-967d07504230 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:35.890724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a5a185b4-a1d6-4802-9062-967d07504230 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:35.953860 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:36.005573 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a185b4-a1d6-4802-9062-967d07504230 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:05:36.007786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a185b4-a1d6-4802-9062-967d07504230 service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:05:36.032052 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 702/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:05:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:05:38.003399 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d54f7-d5e7-4d91-ac12-d5fb5554c61d None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:38.004140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d54f7-d5e7-4d91-ac12-d5fb5554c61d None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:38.004140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d54f7-d5e7-4d91-ac12-d5fb5554c61d None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:38.005667 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 684/1386] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:05:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:38.011545 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e8bc5-0755-4611-86ff-ca32fb72e609 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:38.012088 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e8bc5-0755-4611-86ff-ca32fb72e609 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:38.012549 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e8bc5-0755-4611-86ff-ca32fb72e609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:38.086035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-4b8e8bc5-0755-4611-86ff-ca32fb72e609 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:38.087326 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4b8e8bc5-0755-4611-86ff-ca32fb72e609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:38.189823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 703/1387] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:38.476901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d54f7-d5e7-4d91-ac12-d5fb5554c61d None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:05:38.478524 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-7b5d54f7-d5e7-4d91-ac12-d5fb5554c61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:38.572414 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:38.670294 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d54f7-d5e7-4d91-ac12-d5fb5554c61d None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:05:38.814291 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e10e0-db05-4f6e-afea-09cfd18c38d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:05:38.814817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e10e0-db05-4f6e-afea-09cfd18c38d5 service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:38.815090 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e10e0-db05-4f6e-afea-09cfd18c38d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:38.815352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e10e0-db05-4f6e-afea-09cfd18c38d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:38.816984 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-413e10e0-db05-4f6e-afea-09cfd18c38d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:38.858322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:38.915567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e10e0-db05-4f6e-afea-09cfd18c38d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:05:38.917382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e10e0-db05-4f6e-afea-09cfd18c38d5 service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:05:38.937999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 685/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:05:40.928837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c76b5d-798f-4795-b6fe-50497670df3f None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:40.929385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c76b5d-798f-4795-b6fe-50497670df3f None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:40.929385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c76b5d-798f-4795-b6fe-50497670df3f None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:40.930643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 704/1389] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:40.936450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96014384-3b8f-4975-966e-7b6de991d11d None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:40.936917 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96014384-3b8f-4975-966e-7b6de991d11d None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:40.937048 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96014384-3b8f-4975-966e-7b6de991d11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:41.005762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-96014384-3b8f-4975-966e-7b6de991d11d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:41.007096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-96014384-3b8f-4975-966e-7b6de991d11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:41.109046 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 686/1390] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:41.379709 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b39de-55eb-4648-94b2-f132a6776d05 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:41.380374 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b39de-55eb-4648-94b2-f132a6776d05 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:41.380374 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b39de-55eb-4648-94b2-f132a6776d05 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:41.381815 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 705/1391] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:41.387611 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7db67e-ed96-4d32-aec8-4289cfebb1e1 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:41.387871 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7db67e-ed96-4d32-aec8-4289cfebb1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:41.388148 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7db67e-ed96-4d32-aec8-4289cfebb1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:41.392947 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b39de-55eb-4648-94b2-f132a6776d05 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:05:41.394490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-436b39de-55eb-4648-94b2-f132a6776d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:41.404777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:41.428196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:41.456764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-cd7db67e-ed96-4d32-aec8-4289cfebb1e1 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:41.458000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-cd7db67e-ed96-4d32-aec8-4289cfebb1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:41.546463 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b39de-55eb-4648-94b2-f132a6776d05 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:05:41.559450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 687/1392] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:05:41.654225 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:41.667284 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6c744-e534-4c93-921f-a4143989f5a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:05:41.667945 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6c744-e534-4c93-921f-a4143989f5a2 service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:41.668286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6c744-e534-4c93-921f-a4143989f5a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:41.668733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6c744-e534-4c93-921f-a4143989f5a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:41.670702 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-97d6c744-e534-4c93-921f-a4143989f5a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:41.766177 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d6c744-e534-4c93-921f-a4143989f5a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:05:41.767844 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d6c744-e534-4c93-921f-a4143989f5a2 service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:05:41.786984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 706/1393] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:05:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:05:41.852801 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca652db-f00e-4f4c-944a-2b3c73ab78ac None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:41.853083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca652db-f00e-4f4c-944a-2b3c73ab78ac None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:41.853455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca652db-f00e-4f4c-944a-2b3c73ab78ac None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:41.855009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 688/1394] 10.222.0.17 () {54 vars in 967 bytes} [Tue Aug 23 07:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:41.860606 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51404fab-3801-4c70-a892-ba3fc22bd721 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:41.860958 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51404fab-3801-4c70-a892-ba3fc22bd721 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:41.861228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51404fab-3801-4c70-a892-ba3fc22bd721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:41.955366 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-51404fab-3801-4c70-a892-ba3fc22bd721 None None] MFA Rules not processed for user `57a5f04cb9674b86a6b4b9a472e1afc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:41.956546 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-51404fab-3801-4c70-a892-ba3fc22bd721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:42.016319 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:42.042460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 707/1395] 10.222.0.17 () {58 vars in 1064 bytes} [Tue Aug 23 07:05:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:42.050594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca652db-f00e-4f4c-944a-2b3c73ab78ac None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:05:42.052331 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9ca652db-f00e-4f4c-944a-2b3c73ab78ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:42.157491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:42.165669 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca652db-f00e-4f4c-944a-2b3c73ab78ac None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:05:42.245200 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9739e-5a7e-4418-a7ff-e23cdfbf1af4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57a5f04cb9674b86a6b4b9a472e1afc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:05:42.245730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9739e-5a7e-4418-a7ff-e23cdfbf1af4 service glance] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:42.246009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9739e-5a7e-4418-a7ff-e23cdfbf1af4 service glance] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:42.246365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9739e-5a7e-4418-a7ff-e23cdfbf1af4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:42.248197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f1a9739e-5a7e-4418-a7ff-e23cdfbf1af4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:42.361209 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9739e-5a7e-4418-a7ff-e23cdfbf1af4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:05:42.362670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9739e-5a7e-4418-a7ff-e23cdfbf1af4 service glance] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:05:42.380803 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 689/1396] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:05:43.124741 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20c865-7ff5-4468-95ae-137454231198 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:43.125239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20c865-7ff5-4468-95ae-137454231198 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:43.125343 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20c865-7ff5-4468-95ae-137454231198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:43.192567 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-0a20c865-7ff5-4468-95ae-137454231198 None None] MFA Rules not processed for user `b3f1ae88e29f4172a7db7393d517a893`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:43.193790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-0a20c865-7ff5-4468-95ae-137454231198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:43.285835 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 708/1397] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 23 07:05:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:43.328611 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badc1eb-928a-4908-afd7-6305f3739cf7 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:43.329132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badc1eb-928a-4908-afd7-6305f3739cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:43.329239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badc1eb-928a-4908-afd7-6305f3739cf7 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:43.330504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 690/1398] 10.222.0.17 () {54 vars in 943 bytes} [Tue Aug 23 07:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:43.336829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2857a7-8efe-49a5-a4e0-0f089a6f3559 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:43.337115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2857a7-8efe-49a5-a4e0-0f089a6f3559 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:43.337389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2857a7-8efe-49a5-a4e0-0f089a6f3559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:43.415779 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-ef2857a7-8efe-49a5-a4e0-0f089a6f3559 None None] MFA Rules not processed for user `fd88ec8b477443af95bff9af03471adf`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:43.417005 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ef2857a7-8efe-49a5-a4e0-0f089a6f3559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:43.430612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:43.528474 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 709/1399] 10.222.0.17 () {58 vars in 1040 bytes} [Tue Aug 23 07:05:43 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:43.538096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badc1eb-928a-4908-afd7-6305f3739cf7 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:05:43.539692 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2badc1eb-928a-4908-afd7-6305f3739cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:43.564736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:43.694815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badc1eb-928a-4908-afd7-6305f3739cf7 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:05:43.725535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df37748-a8c4-4e17-bea4-81e79b4c4a16 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:43.725828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df37748-a8c4-4e17-bea4-81e79b4c4a16 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:43.726132 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df37748-a8c4-4e17-bea4-81e79b4c4a16 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:43.727522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 710/1400] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:43.733670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f6399-c696-4f9e-8536-1e65c3491de7 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:43.733949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f6399-c696-4f9e-8536-1e65c3491de7 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:43.734221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f6399-c696-4f9e-8536-1e65c3491de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:43.800833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-954f6399-c696-4f9e-8536-1e65c3491de7 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:43.801716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5035a-bc42-4dd6-ac16-4e7f099d3dee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd88ec8b477443af95bff9af03471adf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:05:43.802359 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5035a-bc42-4dd6-ac16-4e7f099d3dee service swift] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:43.802485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-954f6399-c696-4f9e-8536-1e65c3491de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:43.802632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5035a-bc42-4dd6-ac16-4e7f099d3dee service swift] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:43.802902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5035a-bc42-4dd6-ac16-4e7f099d3dee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:43.804541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-1ce5035a-bc42-4dd6-ac16-4e7f099d3dee service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:43.812119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:43.880333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:43.893047 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5035a-bc42-4dd6-ac16-4e7f099d3dee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:05:43.894706 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5035a-bc42-4dd6-ac16-4e7f099d3dee service swift] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:05:43.903969 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 691/1401] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 23 07:05:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 07:05:43.908826 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 711/1402] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:44.201434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5035a-bc42-4dd6-ac16-4e7f099d3dee service swift] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:05:44.202779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-1ce5035a-bc42-4dd6-ac16-4e7f099d3dee service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:44.333161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5035a-bc42-4dd6-ac16-4e7f099d3dee service swift] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:05:44.400842 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:44.479935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce152e-057b-4c1f-9e96-b9567e09cd91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:05:44.480948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce152e-057b-4c1f-9e96-b9567e09cd91 service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:44.481413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce152e-057b-4c1f-9e96-b9567e09cd91 service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:44.482041 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce152e-057b-4c1f-9e96-b9567e09cd91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:44.483904 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-11ce152e-057b-4c1f-9e96-b9567e09cd91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:44.602769 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce152e-057b-4c1f-9e96-b9567e09cd91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:05:44.606616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce152e-057b-4c1f-9e96-b9567e09cd91 service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:05:44.627284 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 692/1403] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:05:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:05:46.676291 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574e72a-4df0-413e-ae5d-409403166f13 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:46.676869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574e72a-4df0-413e-ae5d-409403166f13 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:46.676869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574e72a-4df0-413e-ae5d-409403166f13 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:46.678194 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 712/1404] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:46.683741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cd2c3-60b0-488d-b872-7db2bcd44ff4 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:46.684828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cd2c3-60b0-488d-b872-7db2bcd44ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:46.685336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cd2c3-60b0-488d-b872-7db2bcd44ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:46.760206 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-449cd2c3-60b0-488d-b872-7db2bcd44ff4 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:46.761721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-449cd2c3-60b0-488d-b872-7db2bcd44ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:46.777501 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:46.870524 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 693/1405] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:47.162368 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574e72a-4df0-413e-ae5d-409403166f13 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:05:47.164010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a574e72a-4df0-413e-ae5d-409403166f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:47.351787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574e72a-4df0-413e-ae5d-409403166f13 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:05:47.431912 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:47.511017 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e92c70-956c-4970-935e-abda89b8b041 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:05:47.511699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e92c70-956c-4970-935e-abda89b8b041 service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:47.513937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e92c70-956c-4970-935e-abda89b8b041 service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:47.514224 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e92c70-956c-4970-935e-abda89b8b041 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:47.516244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-51e92c70-956c-4970-935e-abda89b8b041 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:47.617353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e92c70-956c-4970-935e-abda89b8b041 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:05:47.618886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e92c70-956c-4970-935e-abda89b8b041 service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:05:47.637738 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 713/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:05:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:05:49.633326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec3973-4409-430c-a588-548ce4cbaff7 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:49.633889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec3973-4409-430c-a588-548ce4cbaff7 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:49.633889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec3973-4409-430c-a588-548ce4cbaff7 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:49.635312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 694/1407] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:05:49.641117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576505e-552f-44d4-a9c9-40f6ef66a5a9 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:49.641618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576505e-552f-44d4-a9c9-40f6ef66a5a9 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:49.641712 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576505e-552f-44d4-a9c9-40f6ef66a5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:49.716651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-9576505e-552f-44d4-a9c9-40f6ef66a5a9 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:05:49.717801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9576505e-552f-44d4-a9c9-40f6ef66a5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:49.736943 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:49.828269 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 714/1408] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:05:50.128009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec3973-4409-430c-a588-548ce4cbaff7 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:05:50.129222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-37ec3973-4409-430c-a588-548ce4cbaff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:50.297637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:05:50.300123 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec3973-4409-430c-a588-548ce4cbaff7 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:05:50.423989 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f3324-34f0-436a-8e74-1cf4154b4b8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:05:50.424625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f3324-34f0-436a-8e74-1cf4154b4b8e service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:05:50.424925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f3324-34f0-436a-8e74-1cf4154b4b8e service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:05:50.425157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f3324-34f0-436a-8e74-1cf4154b4b8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:05:50.426794 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b43f3324-34f0-436a-8e74-1cf4154b4b8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:05:50.532050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f3324-34f0-436a-8e74-1cf4154b4b8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:05:50.533817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f3324-34f0-436a-8e74-1cf4154b4b8e service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:05:50.555478 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 695/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:05:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:05:59.826187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=35 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:00.552955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:06.214359 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068de89-a5f9-4ca6-bd09-dc7a72228b65 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:06.214989 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068de89-a5f9-4ca6-bd09-dc7a72228b65 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:06.215121 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068de89-a5f9-4ca6-bd09-dc7a72228b65 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:06.216697 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 715/1410] 10.222.0.17 () {54 vars in 918 bytes} [Tue Aug 23 07:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:06.222214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80def14f-88cc-46fe-8679-ec60dcefa084 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:06.222642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80def14f-88cc-46fe-8679-ec60dcefa084 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:06.222744 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80def14f-88cc-46fe-8679-ec60dcefa084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:06.311221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-80def14f-88cc-46fe-8679-ec60dcefa084 None None] MFA Rules not processed for user `a9b0f54c988e43bda474f978a69fa4fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:06.312950 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-80def14f-88cc-46fe-8679-ec60dcefa084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:06.432124 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 696/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Tue Aug 23 07:06:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:06.446677 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1682719-e62d-4b1c-8aec-c2d52457f501 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:06.446967 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1682719-e62d-4b1c-8aec-c2d52457f501 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:06.447241 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1682719-e62d-4b1c-8aec-c2d52457f501 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:06.448760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 716/1412] 10.222.0.17 () {54 vars in 918 bytes} [Tue Aug 23 07:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:06.453855 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654d5a8-932e-412c-81a6-99040786604e None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:06.454140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654d5a8-932e-412c-81a6-99040786604e None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:06.454406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654d5a8-932e-412c-81a6-99040786604e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:06.531998 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:06.535825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-6654d5a8-932e-412c-81a6-99040786604e None None] MFA Rules not processed for user `a8d2cd93042c4583b63afe629d47fce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:06.538110 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-6654d5a8-932e-412c-81a6-99040786604e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:06.662244 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 697/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Tue Aug 23 07:06:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:09.083179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968252e-9c17-45b6-be26-ac671f9b86db None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:09.083771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968252e-9c17-45b6-be26-ac671f9b86db None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:09.083771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968252e-9c17-45b6-be26-ac671f9b86db None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:09.085072 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 717/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 07:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:09.092887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41bd2b-69cf-4dd0-a7c2-7242936560d7 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:09.093366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41bd2b-69cf-4dd0-a7c2-7242936560d7 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:09.093474 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41bd2b-69cf-4dd0-a7c2-7242936560d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:09.174857 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-0d41bd2b-69cf-4dd0-a7c2-7242936560d7 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:09.176808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-0d41bd2b-69cf-4dd0-a7c2-7242936560d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:09.255128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:09.300498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 698/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 07:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:09.596055 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968252e-9c17-45b6-be26-ac671f9b86db None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:09.598296 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-0968252e-9c17-45b6-be26-ac671f9b86db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:09.756279 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968252e-9c17-45b6-be26-ac671f9b86db None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:09.847406 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:09.889878 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e922c61-6a17-4c57-9db7-58fdc0a4aed9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:09.890474 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e922c61-6a17-4c57-9db7-58fdc0a4aed9 service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:09.890744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e922c61-6a17-4c57-9db7-58fdc0a4aed9 service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:09.891005 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e922c61-6a17-4c57-9db7-58fdc0a4aed9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:09.892649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-1e922c61-6a17-4c57-9db7-58fdc0a4aed9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:09.985985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e922c61-6a17-4c57-9db7-58fdc0a4aed9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:09.987616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e922c61-6a17-4c57-9db7-58fdc0a4aed9 service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:10.005982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 718/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:06:10.895783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88411a00-fb2b-46af-9b28-629047812776 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:10.896380 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88411a00-fb2b-46af-9b28-629047812776 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:10.896380 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88411a00-fb2b-46af-9b28-629047812776 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:10.897687 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 699/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 07:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:10.904763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a7dbf-f036-4dc7-bbe2-8f2e978cc885 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:10.905219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a7dbf-f036-4dc7-bbe2-8f2e978cc885 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:10.905330 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a7dbf-f036-4dc7-bbe2-8f2e978cc885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:11.000707 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-593a7dbf-f036-4dc7-bbe2-8f2e978cc885 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:11.001860 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-593a7dbf-f036-4dc7-bbe2-8f2e978cc885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:11.032511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:11.115261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 719/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 07:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:11.269661 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88411a00-fb2b-46af-9b28-629047812776 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:11.271091 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-88411a00-fb2b-46af-9b28-629047812776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:11.429091 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88411a00-fb2b-46af-9b28-629047812776 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:11.480281 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:11.561680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c104c-9349-44e8-93ad-064c0b33512f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:11.562200 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c104c-9349-44e8-93ad-064c0b33512f service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:11.562475 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c104c-9349-44e8-93ad-064c0b33512f service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:11.562750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c104c-9349-44e8-93ad-064c0b33512f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:11.564398 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-253c104c-9349-44e8-93ad-064c0b33512f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:11.659774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c104c-9349-44e8-93ad-064c0b33512f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:11.661421 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c104c-9349-44e8-93ad-064c0b33512f service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:11.679855 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 700/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:06:12.562713 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa3712-7e66-4f38-b940-3da49158a50f None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:12.563306 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa3712-7e66-4f38-b940-3da49158a50f None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:12.563306 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa3712-7e66-4f38-b940-3da49158a50f None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:12.565794 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 720/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 07:06:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:12.574203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54673509-1118-4dfc-b553-c2007294a7b0 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:12.574653 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54673509-1118-4dfc-b553-c2007294a7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:12.574824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54673509-1118-4dfc-b553-c2007294a7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:12.631774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:12.657740 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-54673509-1118-4dfc-b553-c2007294a7b0 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:12.658997 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-54673509-1118-4dfc-b553-c2007294a7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:12.763558 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 701/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 07:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:12.914992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa3712-7e66-4f38-b940-3da49158a50f None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:12.916585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-afaa3712-7e66-4f38-b940-3da49158a50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:13.083902 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa3712-7e66-4f38-b940-3da49158a50f None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:13.097930 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:13.214227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c38da-0f1c-4430-ac34-b2305b1f8c5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:13.214843 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c38da-0f1c-4430-ac34-b2305b1f8c5a service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:13.215010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c38da-0f1c-4430-ac34-b2305b1f8c5a service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:13.215439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c38da-0f1c-4430-ac34-b2305b1f8c5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:13.217087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-388c38da-0f1c-4430-ac34-b2305b1f8c5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:13.312911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c38da-0f1c-4430-ac34-b2305b1f8c5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:13.314530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c38da-0f1c-4430-ac34-b2305b1f8c5a service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:13.334324 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 721/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:06:14.190078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519e228-82ce-4fdb-97c2-711231df0637 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:14.190886 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519e228-82ce-4fdb-97c2-711231df0637 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:14.190886 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519e228-82ce-4fdb-97c2-711231df0637 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:14.192357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 702/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 07:06:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:14.198454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1927a3-8fbb-44e5-a4a2-73e3678742ee None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:14.198917 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1927a3-8fbb-44e5-a4a2-73e3678742ee None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:14.199334 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1927a3-8fbb-44e5-a4a2-73e3678742ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:14.226769 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:14.287878 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-8b1927a3-8fbb-44e5-a4a2-73e3678742ee None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:14.289105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8b1927a3-8fbb-44e5-a4a2-73e3678742ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:14.401473 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 722/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 07:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:14.557048 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519e228-82ce-4fdb-97c2-711231df0637 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:14.559689 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-1519e228-82ce-4fdb-97c2-711231df0637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:14.686631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:14.725290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519e228-82ce-4fdb-97c2-711231df0637 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:14.845167 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513665f2-c738-4626-915f-b0b26eb68a76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:14.845776 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513665f2-c738-4626-915f-b0b26eb68a76 service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:14.846049 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513665f2-c738-4626-915f-b0b26eb68a76 service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:14.846315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513665f2-c738-4626-915f-b0b26eb68a76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:14.847960 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-513665f2-c738-4626-915f-b0b26eb68a76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:14.927234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:14.942499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513665f2-c738-4626-915f-b0b26eb68a76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:14.944151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513665f2-c738-4626-915f-b0b26eb68a76 service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:14.962212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 703/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:06:15.862806 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4a484-ca4f-48de-9c38-605647522018 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:15.863378 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4a484-ca4f-48de-9c38-605647522018 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:15.863378 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4a484-ca4f-48de-9c38-605647522018 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:15.864765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 723/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 07:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:15.871107 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6901e-c7b4-4762-92f3-eec1bedff38e None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:15.871572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6901e-c7b4-4762-92f3-eec1bedff38e None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:15.871889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6901e-c7b4-4762-92f3-eec1bedff38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:15.954756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-f5c6901e-c7b4-4762-92f3-eec1bedff38e None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:15.955922 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f5c6901e-c7b4-4762-92f3-eec1bedff38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:16.065167 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 704/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 07:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:16.219138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4a484-ca4f-48de-9c38-605647522018 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:16.220597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a2b4a484-ca4f-48de-9c38-605647522018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:16.312707 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:16.378124 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4a484-ca4f-48de-9c38-605647522018 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:16.512749 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e97a1-fe75-4fad-92ee-2cafdc5a006e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:16.513267 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e97a1-fe75-4fad-92ee-2cafdc5a006e service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:16.513546 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e97a1-fe75-4fad-92ee-2cafdc5a006e service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:16.513816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e97a1-fe75-4fad-92ee-2cafdc5a006e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:16.515445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b17e97a1-fe75-4fad-92ee-2cafdc5a006e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:16.568790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:16.612865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e97a1-fe75-4fad-92ee-2cafdc5a006e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:16.614747 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e97a1-fe75-4fad-92ee-2cafdc5a006e service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:16.633489 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 724/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:06:17.440893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aeada5-0833-454f-ab27-a871dbc409f5 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:17.441819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aeada5-0833-454f-ab27-a871dbc409f5 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:17.441819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aeada5-0833-454f-ab27-a871dbc409f5 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:17.444022 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 705/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 07:06:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:17.450852 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4900d3-0f12-40a6-8a24-dc52a2507b53 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:17.451621 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4900d3-0f12-40a6-8a24-dc52a2507b53 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:17.451790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4900d3-0f12-40a6-8a24-dc52a2507b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:17.542112 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-6c4900d3-0f12-40a6-8a24-dc52a2507b53 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:17.543645 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6c4900d3-0f12-40a6-8a24-dc52a2507b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:17.661290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 725/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 07:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:17.819704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aeada5-0833-454f-ab27-a871dbc409f5 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:17.821263 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-58aeada5-0833-454f-ab27-a871dbc409f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:17.879510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:17.986154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aeada5-0833-454f-ab27-a871dbc409f5 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:18.108770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b57a3-f206-4b07-b85e-463a4d8472c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:18.109383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b57a3-f206-4b07-b85e-463a4d8472c6 service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:18.109664 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b57a3-f206-4b07-b85e-463a4d8472c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:18.109936 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b57a3-f206-4b07-b85e-463a4d8472c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:18.111567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8d8b57a3-f206-4b07-b85e-463a4d8472c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:18.118725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:18.221505 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b57a3-f206-4b07-b85e-463a4d8472c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:18.223119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b57a3-f206-4b07-b85e-463a4d8472c6 service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:18.247055 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 706/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:06:19.093060 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075fd29-5659-4355-9a47-ab4c11dc378a None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:19.093660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075fd29-5659-4355-9a47-ab4c11dc378a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:19.093660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075fd29-5659-4355-9a47-ab4c11dc378a None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:19.095653 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 726/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 07:06:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:19.103989 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f13ad-8469-4b9a-8c88-fd5d46171f3a None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:19.104435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f13ad-8469-4b9a-8c88-fd5d46171f3a None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:19.104533 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f13ad-8469-4b9a-8c88-fd5d46171f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:19.191181 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-638f13ad-8469-4b9a-8c88-fd5d46171f3a None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:19.192437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-638f13ad-8469-4b9a-8c88-fd5d46171f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:19.282902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:19.316015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 707/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 07:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:19.474106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075fd29-5659-4355-9a47-ab4c11dc378a None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:19.475736 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4075fd29-5659-4355-9a47-ab4c11dc378a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:19.494412 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:19.671508 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075fd29-5659-4355-9a47-ab4c11dc378a None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:19.773679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:19.803124 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaca61b-7ae9-4392-afcf-fe18f8a53847 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:19.803768 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaca61b-7ae9-4392-afcf-fe18f8a53847 service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:19.804039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaca61b-7ae9-4392-afcf-fe18f8a53847 service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:19.804308 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaca61b-7ae9-4392-afcf-fe18f8a53847 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:19.805927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-deaca61b-7ae9-4392-afcf-fe18f8a53847 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:19.902475 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaca61b-7ae9-4392-afcf-fe18f8a53847 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:19.904161 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaca61b-7ae9-4392-afcf-fe18f8a53847 service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:19.923462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 727/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:06:20.728222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70919137-ea0b-4304-897f-97ab06aa984f None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:20.728770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70919137-ea0b-4304-897f-97ab06aa984f None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:20.728770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70919137-ea0b-4304-897f-97ab06aa984f None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:20.730165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 708/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 07:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:20.736337 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003459b4-e673-4d0b-b2d9-eb6cb073d62e None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:20.737108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003459b4-e673-4d0b-b2d9-eb6cb073d62e None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:20.737266 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003459b4-e673-4d0b-b2d9-eb6cb073d62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:20.825008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-003459b4-e673-4d0b-b2d9-eb6cb073d62e None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:20.826596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-003459b4-e673-4d0b-b2d9-eb6cb073d62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:20.874187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:20.934430 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 728/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 07:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:21.091125 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70919137-ea0b-4304-897f-97ab06aa984f None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:21.092724 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-70919137-ea0b-4304-897f-97ab06aa984f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:21.262963 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70919137-ea0b-4304-897f-97ab06aa984f None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:21.318811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:21.381072 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290aca6-5600-4d38-abcc-77e1c0b5acc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:21.381677 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290aca6-5600-4d38-abcc-77e1c0b5acc0 service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:21.381949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290aca6-5600-4d38-abcc-77e1c0b5acc0 service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:21.382215 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290aca6-5600-4d38-abcc-77e1c0b5acc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:21.383876 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-4290aca6-5600-4d38-abcc-77e1c0b5acc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:21.481310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290aca6-5600-4d38-abcc-77e1c0b5acc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:21.482936 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290aca6-5600-4d38-abcc-77e1c0b5acc0 service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:21.500742 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 709/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:06:22.323249 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef80f78-ec79-438a-840c-e60c5fa74e44 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:22.323918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef80f78-ec79-438a-840c-e60c5fa74e44 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:22.323918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef80f78-ec79-438a-840c-e60c5fa74e44 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:22.325215 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 729/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 07:06:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:22.331499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1bf81-25c7-46a5-a260-a63df030466e None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:22.331919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1bf81-25c7-46a5-a260-a63df030466e None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:22.332108 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1bf81-25c7-46a5-a260-a63df030466e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:22.419316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-a6e1bf81-25c7-46a5-a260-a63df030466e None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:22.420555 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-a6e1bf81-25c7-46a5-a260-a63df030466e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:22.422671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:22.518802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 710/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 07:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:22.670307 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef80f78-ec79-438a-840c-e60c5fa74e44 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:22.672734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-5ef80f78-ec79-438a-840c-e60c5fa74e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:22.830454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef80f78-ec79-438a-840c-e60c5fa74e44 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:22.859928 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:22.956418 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58121a-f86b-4007-a68b-453abd5f275a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:22.956925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58121a-f86b-4007-a68b-453abd5f275a service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:22.957201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58121a-f86b-4007-a68b-453abd5f275a service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:22.957498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58121a-f86b-4007-a68b-453abd5f275a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:22.959085 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-2a58121a-f86b-4007-a68b-453abd5f275a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:23.054527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58121a-f86b-4007-a68b-453abd5f275a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:23.056717 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58121a-f86b-4007-a68b-453abd5f275a service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:23.081656 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 730/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:06:24.883222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589afd1-038c-486c-aed6-b30b8eee8168 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:24.883855 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589afd1-038c-486c-aed6-b30b8eee8168 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:24.883855 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589afd1-038c-486c-aed6-b30b8eee8168 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:24.885142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 711/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 07:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:24.893697 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f79ab-815b-458b-a6f5-dd42843fe516 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:24.894165 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f79ab-815b-458b-a6f5-dd42843fe516 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:24.894269 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f79ab-815b-458b-a6f5-dd42843fe516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:24.954568 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:24.985993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-0c8f79ab-815b-458b-a6f5-dd42843fe516 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:24.987271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-0c8f79ab-815b-458b-a6f5-dd42843fe516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:25.097806 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 731/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 07:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:25.387777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589afd1-038c-486c-aed6-b30b8eee8168 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:25.390194 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-4589afd1-038c-486c-aed6-b30b8eee8168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:25.500098 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589afd1-038c-486c-aed6-b30b8eee8168 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:25.540008 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:25.586164 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765ffaa-36ea-4f16-9c76-ec8ae6ffbef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:25.586820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765ffaa-36ea-4f16-9c76-ec8ae6ffbef4 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:25.587141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765ffaa-36ea-4f16-9c76-ec8ae6ffbef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:25.587447 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765ffaa-36ea-4f16-9c76-ec8ae6ffbef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:25.589172 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e765ffaa-36ea-4f16-9c76-ec8ae6ffbef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:25.714151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e765ffaa-36ea-4f16-9c76-ec8ae6ffbef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:25.716245 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e765ffaa-36ea-4f16-9c76-ec8ae6ffbef4 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:25.735801 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 712/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:06:27.787226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c6d07-4700-448c-8900-32322823f807 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:27.787811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c6d07-4700-448c-8900-32322823f807 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:27.787811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c6d07-4700-448c-8900-32322823f807 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:27.789180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 732/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 07:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:27.794783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd230122-3aa7-416e-ad2b-0d2e8cc6ca05 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:27.795211 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd230122-3aa7-416e-ad2b-0d2e8cc6ca05 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:27.795315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd230122-3aa7-416e-ad2b-0d2e8cc6ca05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:27.868479 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-bd230122-3aa7-416e-ad2b-0d2e8cc6ca05 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:27.871553 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-bd230122-3aa7-416e-ad2b-0d2e8cc6ca05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:27.886349 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:27.987034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 713/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 07:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:28.266450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c6d07-4700-448c-8900-32322823f807 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:28.268031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-748c6d07-4700-448c-8900-32322823f807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:28.390532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c6d07-4700-448c-8900-32322823f807 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:28.403631 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:28.469371 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f822905-b4cb-42a2-9413-31e63e344ed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:28.469978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f822905-b4cb-42a2-9413-31e63e344ed4 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:28.470264 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f822905-b4cb-42a2-9413-31e63e344ed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:28.470536 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f822905-b4cb-42a2-9413-31e63e344ed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:28.472189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4f822905-b4cb-42a2-9413-31e63e344ed4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:28.602033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f822905-b4cb-42a2-9413-31e63e344ed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:28.603373 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f822905-b4cb-42a2-9413-31e63e344ed4 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:28.622901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 733/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:06:30.862860 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5fc5f-44ef-4e3c-8316-19bda91449c0 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:30.863654 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5fc5f-44ef-4e3c-8316-19bda91449c0 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:30.863654 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5fc5f-44ef-4e3c-8316-19bda91449c0 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:30.865657 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 714/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 07:06:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:30.871530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8dec2-ca7e-4628-936d-5abb9c2dc8fa None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:30.871993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8dec2-ca7e-4628-936d-5abb9c2dc8fa None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:30.872193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8dec2-ca7e-4628-936d-5abb9c2dc8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:30.926801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:30.947337 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-1fc8dec2-ca7e-4628-936d-5abb9c2dc8fa None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:30.948606 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-1fc8dec2-ca7e-4628-936d-5abb9c2dc8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:31.062513 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 734/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 07:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:31.352032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5fc5f-44ef-4e3c-8316-19bda91449c0 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:31.354340 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9fd5fc5f-44ef-4e3c-8316-19bda91449c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:31.464338 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5fc5f-44ef-4e3c-8316-19bda91449c0 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:31.507310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:31.543730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89304a-9078-43aa-9cba-f079ec2ba2cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:31.544341 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89304a-9078-43aa-9cba-f079ec2ba2cf service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:31.544615 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89304a-9078-43aa-9cba-f079ec2ba2cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:31.544880 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89304a-9078-43aa-9cba-f079ec2ba2cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:31.546471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3a89304a-9078-43aa-9cba-f079ec2ba2cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:31.661155 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a89304a-9078-43aa-9cba-f079ec2ba2cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:31.662737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a89304a-9078-43aa-9cba-f079ec2ba2cf service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:31.688050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 715/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:06:33.772620 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a13fd-673a-4938-ad0d-ac5f3e255a52 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:33.773518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a13fd-673a-4938-ad0d-ac5f3e255a52 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:33.773518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a13fd-673a-4938-ad0d-ac5f3e255a52 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:33.775682 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 735/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 07:06:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:33.782293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6197d4-5a68-4d1a-bbdd-797450b47285 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:33.782749 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6197d4-5a68-4d1a-bbdd-797450b47285 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:33.782838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6197d4-5a68-4d1a-bbdd-797450b47285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:33.857461 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-ec6197d4-5a68-4d1a-bbdd-797450b47285 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:33.858709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ec6197d4-5a68-4d1a-bbdd-797450b47285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:33.881865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:33.967737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 716/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 07:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:34.247393 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a13fd-673a-4938-ad0d-ac5f3e255a52 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:34.249892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a27a13fd-673a-4938-ad0d-ac5f3e255a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:34.377729 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a13fd-673a-4938-ad0d-ac5f3e255a52 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:34.400811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:34.464886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ab932-833e-436a-a609-e7738124a056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:34.465756 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ab932-833e-436a-a609-e7738124a056 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:34.466031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ab932-833e-436a-a609-e7738124a056 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:34.466301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ab932-833e-436a-a609-e7738124a056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:34.468308 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-183ab932-833e-436a-a609-e7738124a056 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:34.585434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ab932-833e-436a-a609-e7738124a056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:34.586801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ab932-833e-436a-a609-e7738124a056 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:34.607048 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 736/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:06:36.546873 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544173c-833b-4584-8807-3d0e698c6033 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:36.547492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544173c-833b-4584-8807-3d0e698c6033 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:36.547492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544173c-833b-4584-8807-3d0e698c6033 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:36.548795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 717/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 07:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:36.554396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52996b-d513-4da3-b269-843fc6de1eea None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:36.554870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52996b-d513-4da3-b269-843fc6de1eea None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:36.554965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52996b-d513-4da3-b269-843fc6de1eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:36.626677 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:36.629924 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-6a52996b-d513-4da3-b269-843fc6de1eea None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:36.632205 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6a52996b-d513-4da3-b269-843fc6de1eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:36.732485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 737/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 07:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:37.015428 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544173c-833b-4584-8807-3d0e698c6033 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:37.017083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9544173c-833b-4584-8807-3d0e698c6033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:37.134338 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544173c-833b-4584-8807-3d0e698c6033 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:37.188583 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:37.215429 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46f231-9158-4cb9-9b30-3e33886e2aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:37.216061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46f231-9158-4cb9-9b30-3e33886e2aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:37.216359 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46f231-9158-4cb9-9b30-3e33886e2aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:37.216629 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46f231-9158-4cb9-9b30-3e33886e2aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:37.218251 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-4b46f231-9158-4cb9-9b30-3e33886e2aa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:37.347492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46f231-9158-4cb9-9b30-3e33886e2aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:37.348972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46f231-9158-4cb9-9b30-3e33886e2aa0 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:37.368975 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 718/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:06:39.336039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ccb9f-be4c-4fbd-a0cd-4e6615c57d47 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:39.336848 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ccb9f-be4c-4fbd-a0cd-4e6615c57d47 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:39.336848 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ccb9f-be4c-4fbd-a0cd-4e6615c57d47 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:39.338941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 738/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 07:06:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:39.344762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403624a7-e425-438f-bcd9-bf5a5b68888c None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:39.345202 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403624a7-e425-438f-bcd9-bf5a5b68888c None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:39.345297 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403624a7-e425-438f-bcd9-bf5a5b68888c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:39.421035 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-403624a7-e425-438f-bcd9-bf5a5b68888c None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:39.422258 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-403624a7-e425-438f-bcd9-bf5a5b68888c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:39.454020 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:39.524240 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 719/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 07:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:39.827464 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ccb9f-be4c-4fbd-a0cd-4e6615c57d47 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:39.829865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-769ccb9f-be4c-4fbd-a0cd-4e6615c57d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:39.946375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ccb9f-be4c-4fbd-a0cd-4e6615c57d47 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:39.977880 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:40.025739 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd1db9-13a0-4282-bde4-503054fa585c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:40.026348 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd1db9-13a0-4282-bde4-503054fa585c service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:40.026629 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd1db9-13a0-4282-bde4-503054fa585c service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:40.027082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd1db9-13a0-4282-bde4-503054fa585c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:40.028793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-04bd1db9-13a0-4282-bde4-503054fa585c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:40.152909 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd1db9-13a0-4282-bde4-503054fa585c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:40.154371 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd1db9-13a0-4282-bde4-503054fa585c service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:40.173862 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 739/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:06:42.106709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e2763-b6b5-45a2-b774-7211a9be28c9 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:42.107208 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e2763-b6b5-45a2-b774-7211a9be28c9 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:42.107337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e2763-b6b5-45a2-b774-7211a9be28c9 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:42.108735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 720/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 07:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:42.114340 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c356b7-b0e8-4f05-9fc5-6c3bcb604ac4 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:42.114805 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c356b7-b0e8-4f05-9fc5-6c3bcb604ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:42.114911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c356b7-b0e8-4f05-9fc5-6c3bcb604ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:42.188587 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-12c356b7-b0e8-4f05-9fc5-6c3bcb604ac4 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:42.190529 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-12c356b7-b0e8-4f05-9fc5-6c3bcb604ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:42.205254 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:42.306113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 740/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 07:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:42.598469 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e2763-b6b5-45a2-b774-7211a9be28c9 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:42.601005 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-235e2763-b6b5-45a2-b774-7211a9be28c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:42.713843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e2763-b6b5-45a2-b774-7211a9be28c9 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:42.783486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:42.804778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d0103-601c-417f-9772-a09a2342dc1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:42.805397 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d0103-601c-417f-9772-a09a2342dc1b service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:42.805680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d0103-601c-417f-9772-a09a2342dc1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:42.805959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d0103-601c-417f-9772-a09a2342dc1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:42.807613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-309d0103-601c-417f-9772-a09a2342dc1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:42.933232 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d0103-601c-417f-9772-a09a2342dc1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:42.934654 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d0103-601c-417f-9772-a09a2342dc1b service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:42.953633 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 721/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:06:44.924778 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b551a-9ffa-4906-ae72-91ea80d3e7d7 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:44.925614 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b551a-9ffa-4906-ae72-91ea80d3e7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:44.925614 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b551a-9ffa-4906-ae72-91ea80d3e7d7 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:44.927753 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 741/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 07:06:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:44.933477 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa862685-df23-4610-a216-7e3cab0fcf5f None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:44.933929 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa862685-df23-4610-a216-7e3cab0fcf5f None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:44.934040 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa862685-df23-4610-a216-7e3cab0fcf5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:45.002129 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-aa862685-df23-4610-a216-7e3cab0fcf5f None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:45.003333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-aa862685-df23-4610-a216-7e3cab0fcf5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:45.042567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:45.108985 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 722/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 07:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:45.389003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b551a-9ffa-4906-ae72-91ea80d3e7d7 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:45.390563 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-379b551a-9ffa-4906-ae72-91ea80d3e7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:45.505238 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b551a-9ffa-4906-ae72-91ea80d3e7d7 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:45.547080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:45.594565 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a877fdf-f452-4914-b3fd-a821e691cd4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:45.595147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a877fdf-f452-4914-b3fd-a821e691cd4d service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:45.595442 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a877fdf-f452-4914-b3fd-a821e691cd4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:45.595757 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a877fdf-f452-4914-b3fd-a821e691cd4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:45.597354 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6a877fdf-f452-4914-b3fd-a821e691cd4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:45.710258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a877fdf-f452-4914-b3fd-a821e691cd4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:45.711768 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a877fdf-f452-4914-b3fd-a821e691cd4d service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:45.731885 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 742/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:06:47.626391 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4c5ef-bf56-45e5-913a-556adfcd9ccd None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:47.626957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4c5ef-bf56-45e5-913a-556adfcd9ccd None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:47.626957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4c5ef-bf56-45e5-913a-556adfcd9ccd None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:47.628371 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 723/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 07:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:47.634815 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47dc5a-8c65-4fa9-a8e7-cb3d907c9803 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:47.635277 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47dc5a-8c65-4fa9-a8e7-cb3d907c9803 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:47.635379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47dc5a-8c65-4fa9-a8e7-cb3d907c9803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:47.716334 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-9f47dc5a-8c65-4fa9-a8e7-cb3d907c9803 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:47.717570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9f47dc5a-8c65-4fa9-a8e7-cb3d907c9803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:47.736716 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:47.828073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 743/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 07:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:48.113718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4c5ef-bf56-45e5-913a-556adfcd9ccd None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:48.116197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-79c4c5ef-bf56-45e5-913a-556adfcd9ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:48.224910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4c5ef-bf56-45e5-913a-556adfcd9ccd None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:48.310755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:48.318751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794ee68-3071-432b-ae87-1b5d7c74524c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:48.319356 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794ee68-3071-432b-ae87-1b5d7c74524c service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:48.319658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794ee68-3071-432b-ae87-1b5d7c74524c service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:48.319928 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794ee68-3071-432b-ae87-1b5d7c74524c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:48.321572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-7794ee68-3071-432b-ae87-1b5d7c74524c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:48.445367 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794ee68-3071-432b-ae87-1b5d7c74524c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:48.446813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794ee68-3071-432b-ae87-1b5d7c74524c service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:48.466610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 724/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:06:50.358477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7b542-a2f6-4c2c-a011-b3ffa36565ab None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:50.359319 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7b542-a2f6-4c2c-a011-b3ffa36565ab None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:50.359319 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7b542-a2f6-4c2c-a011-b3ffa36565ab None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:50.361448 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 744/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 07:06:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:06:50.368017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d631de-498f-4d5e-9a8f-4b4d265149a9 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:50.368657 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d631de-498f-4d5e-9a8f-4b4d265149a9 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:50.368784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d631de-498f-4d5e-9a8f-4b4d265149a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:50.439544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-d9d631de-498f-4d5e-9a8f-4b4d265149a9 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:06:50.440768 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d9d631de-498f-4d5e-9a8f-4b4d265149a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:50.496913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:50.549256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 725/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 07:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:06:50.837549 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc7b542-a2f6-4c2c-a011-b3ffa36565ab None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:06:50.840915 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ebc7b542-a2f6-4c2c-a011-b3ffa36565ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:50.957032 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc7b542-a2f6-4c2c-a011-b3ffa36565ab None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:06:51.015477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:06:51.044452 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39664e-33da-46e1-ae8d-544945bb50f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:06:51.045058 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39664e-33da-46e1-ae8d-544945bb50f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:06:51.045332 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39664e-33da-46e1-ae8d-544945bb50f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:06:51.045609 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39664e-33da-46e1-ae8d-544945bb50f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:06:51.047195 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-cf39664e-33da-46e1-ae8d-544945bb50f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:06:51.172854 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39664e-33da-46e1-ae8d-544945bb50f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:06:51.174289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39664e-33da-46e1-ae8d-544945bb50f6 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:06:51.194500 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 745/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:07:00.547100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=22 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:07:01.191443 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=66 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:07:10.894803 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9f87d-e341-421c-a355-2bfbacc94c1d None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:10.895718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9f87d-e341-421c-a355-2bfbacc94c1d None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:10.895718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9f87d-e341-421c-a355-2bfbacc94c1d None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:10.897837 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 726/1471] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:07:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:07:10.905899 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6198ec-7855-443b-a3db-e07807224881 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:10.906383 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6198ec-7855-443b-a3db-e07807224881 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:10.906501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6198ec-7855-443b-a3db-e07807224881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:10.982132 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-9b6198ec-7855-443b-a3db-e07807224881 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:07:10.983379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9b6198ec-7855-443b-a3db-e07807224881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:11.100734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 746/1472] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:07:11.420670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0233a7a-f7b9-4372-91b5-f38e79658a1a None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:11.421040 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0233a7a-f7b9-4372-91b5-f38e79658a1a None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:11.421401 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0233a7a-f7b9-4372-91b5-f38e79658a1a None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:11.423440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 727/1473] 10.222.0.17 () {54 vars in 967 bytes} [Tue Aug 23 07:07:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:07:11.430789 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f5256-db27-457c-a9c8-44f417cce500 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:11.431158 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f5256-db27-457c-a9c8-44f417cce500 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:11.431495 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f5256-db27-457c-a9c8-44f417cce500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:11.511653 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-cf9f5256-db27-457c-a9c8-44f417cce500 None None] MFA Rules not processed for user `57a5f04cb9674b86a6b4b9a472e1afc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:07:11.512899 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-cf9f5256-db27-457c-a9c8-44f417cce500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:11.533868 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:07:11.597491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 747/1474] 10.222.0.17 () {58 vars in 1064 bytes} [Tue Aug 23 07:07:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:07:11.607363 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0233a7a-f7b9-4372-91b5-f38e79658a1a None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:07:11.609835 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f0233a7a-f7b9-4372-91b5-f38e79658a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:11.731974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0233a7a-f7b9-4372-91b5-f38e79658a1a None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:07:11.808611 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfa01b-d754-40a9-bc6d-1e3f2af7ef6c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57a5f04cb9674b86a6b4b9a472e1afc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:07:11.809218 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfa01b-d754-40a9-bc6d-1e3f2af7ef6c service glance] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:11.809498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfa01b-d754-40a9-bc6d-1e3f2af7ef6c service glance] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:11.809763 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfa01b-d754-40a9-bc6d-1e3f2af7ef6c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:11.811351 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9ecfa01b-d754-40a9-bc6d-1e3f2af7ef6c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:11.869711 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:07:11.931143 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfa01b-d754-40a9-bc6d-1e3f2af7ef6c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:07:11.932610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfa01b-d754-40a9-bc6d-1e3f2af7ef6c service glance] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:07:11.952542 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 728/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:07:13.927934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc7cdd-6230-4a71-9dca-b7aeadb99a47 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:13.928486 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc7cdd-6230-4a71-9dca-b7aeadb99a47 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:13.928486 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc7cdd-6230-4a71-9dca-b7aeadb99a47 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:13.929741 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 748/1476] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:07:13.936346 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ed6f6-3c90-4363-b8b5-174143112141 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:13.937067 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ed6f6-3c90-4363-b8b5-174143112141 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:13.937265 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ed6f6-3c90-4363-b8b5-174143112141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:14.016194 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-c11ed6f6-3c90-4363-b8b5-174143112141 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:07:14.017431 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c11ed6f6-3c90-4363-b8b5-174143112141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:14.121971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 729/1477] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:07:14.391293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc7cdd-6230-4a71-9dca-b7aeadb99a47 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:07:14.392917 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-76bc7cdd-6230-4a71-9dca-b7aeadb99a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:14.539394 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc7cdd-6230-4a71-9dca-b7aeadb99a47 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:07:14.594132 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:07:14.640699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3fa56-1138-443a-befc-8b02781db9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:07:14.641458 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3fa56-1138-443a-befc-8b02781db9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:14.641634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3fa56-1138-443a-befc-8b02781db9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:14.641947 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3fa56-1138-443a-befc-8b02781db9d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:14.642585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3fa56-1138-443a-befc-8b02781db9d4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:07:14.644488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: WARNING py.warnings [None req-f4c3fa56-1138-443a-befc-8b02781db9d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 07:07:14.644488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: warnings.warn(msg) Aug 23 07:07:14.644488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: Aug 23 07:07:14.645643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3fa56-1138-443a-befc-8b02781db9d4 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:07:14.665063 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 749/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Aug 23 07:07:14 2022] GET /identity/v3/projects => generated 2553 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:07:16.572705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5f774-d0ea-4841-8d44-7bf6f8777055 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:16.573326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5f774-d0ea-4841-8d44-7bf6f8777055 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:16.573326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5f774-d0ea-4841-8d44-7bf6f8777055 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:16.574761 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 730/1479] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:07:16.581267 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def176ed-db3d-4340-9272-486e145f4e2a None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:16.582040 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def176ed-db3d-4340-9272-486e145f4e2a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:16.582184 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def176ed-db3d-4340-9272-486e145f4e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:16.671127 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-def176ed-db3d-4340-9272-486e145f4e2a None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:07:16.672804 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-def176ed-db3d-4340-9272-486e145f4e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:16.779441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 750/1480] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:07:17.064828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5f774-d0ea-4841-8d44-7bf6f8777055 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:07:17.066582 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-acd5f774-d0ea-4841-8d44-7bf6f8777055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:17.077042 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:07:17.226797 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5f774-d0ea-4841-8d44-7bf6f8777055 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:07:17.324256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:07:17.364272 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a7634-9259-467b-b41c-8829c5ce8bc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:07:17.364883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a7634-9259-467b-b41c-8829c5ce8bc8 service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:17.365172 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a7634-9259-467b-b41c-8829c5ce8bc8 service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:17.365422 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a7634-9259-467b-b41c-8829c5ce8bc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:17.367043 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-054a7634-9259-467b-b41c-8829c5ce8bc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:17.467816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a7634-9259-467b-b41c-8829c5ce8bc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:07:17.469452 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a7634-9259-467b-b41c-8829c5ce8bc8 service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:07:17.493572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 731/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:07:19.629091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c02b2-8f1c-4e1d-b67d-1bc878cff830 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:19.629985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c02b2-8f1c-4e1d-b67d-1bc878cff830 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:19.629985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c02b2-8f1c-4e1d-b67d-1bc878cff830 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:19.632174 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 751/1482] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:07:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:07:19.640274 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd590ac2-c983-4a64-abe6-de0c69e1692d None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:19.640744 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd590ac2-c983-4a64-abe6-de0c69e1692d None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:19.640840 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd590ac2-c983-4a64-abe6-de0c69e1692d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:19.722242 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-dd590ac2-c983-4a64-abe6-de0c69e1692d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:07:19.723454 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-dd590ac2-c983-4a64-abe6-de0c69e1692d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:19.770613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:07:19.828569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 732/1483] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:07:20.116768 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c02b2-8f1c-4e1d-b67d-1bc878cff830 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:07:20.119186 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-da1c02b2-8f1c-4e1d-b67d-1bc878cff830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:20.151371 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:07:20.369071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c02b2-8f1c-4e1d-b67d-1bc878cff830 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:07:20.447322 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3936cdd-d0b5-43a3-9561-f834bfb46fc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:07:20.448017 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3936cdd-d0b5-43a3-9561-f834bfb46fc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:20.448329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3936cdd-d0b5-43a3-9561-f834bfb46fc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:20.448641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3936cdd-d0b5-43a3-9561-f834bfb46fc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:20.450260 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b3936cdd-d0b5-43a3-9561-f834bfb46fc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:20.531510 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:07:20.571712 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3936cdd-d0b5-43a3-9561-f834bfb46fc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:07:20.573177 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3936cdd-d0b5-43a3-9561-f834bfb46fc5 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:07:20.591788 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 752/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:07:22.629200 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449b64c-7b5a-48d8-957d-6c2be0e8dd24 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:22.630211 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449b64c-7b5a-48d8-957d-6c2be0e8dd24 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:22.630211 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449b64c-7b5a-48d8-957d-6c2be0e8dd24 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:22.632285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 733/1485] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:07:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:07:22.638383 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f3b23-ec5a-4b7a-a5b7-0620a709fa4f None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:22.638853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f3b23-ec5a-4b7a-a5b7-0620a709fa4f None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:22.638989 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f3b23-ec5a-4b7a-a5b7-0620a709fa4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:22.710992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-b26f3b23-ec5a-4b7a-a5b7-0620a709fa4f None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:07:22.712305 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b26f3b23-ec5a-4b7a-a5b7-0620a709fa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:22.816899 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 753/1486] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:07:23.115501 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449b64c-7b5a-48d8-957d-6c2be0e8dd24 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:07:23.117997 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3449b64c-7b5a-48d8-957d-6c2be0e8dd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:23.240194 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449b64c-7b5a-48d8-957d-6c2be0e8dd24 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:07:23.319256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:07:23.321536 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3c18e-9697-4f9d-8186-fadf7ca223eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:07:23.322152 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3c18e-9697-4f9d-8186-fadf7ca223eb service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:23.322439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3c18e-9697-4f9d-8186-fadf7ca223eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:23.322736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3c18e-9697-4f9d-8186-fadf7ca223eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:23.325781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-afb3c18e-9697-4f9d-8186-fadf7ca223eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:23.581170 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3c18e-9697-4f9d-8186-fadf7ca223eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:07:23.582777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3c18e-9697-4f9d-8186-fadf7ca223eb service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:07:23.602351 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 734/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:07:25.578364 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581c471-a622-4d01-892d-3224f5b9c322 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:25.578969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581c471-a622-4d01-892d-3224f5b9c322 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:25.578969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581c471-a622-4d01-892d-3224f5b9c322 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:25.580295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 754/1488] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:07:25.585961 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10974e66-c5b6-4fca-9147-d0573bb61f2b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:25.586400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10974e66-c5b6-4fca-9147-d0573bb61f2b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:25.586512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10974e66-c5b6-4fca-9147-d0573bb61f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:25.665632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-10974e66-c5b6-4fca-9147-d0573bb61f2b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:07:25.666858 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-10974e66-c5b6-4fca-9147-d0573bb61f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:25.726428 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:07:25.787830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 735/1489] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:07:26.076897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581c471-a622-4d01-892d-3224f5b9c322 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:07:26.079530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-7581c471-a622-4d01-892d-3224f5b9c322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:26.119320 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:07:26.196343 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581c471-a622-4d01-892d-3224f5b9c322 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:07:26.277776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210f8b9-e75b-4664-b405-2a6f4435810b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:07:26.278372 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210f8b9-e75b-4664-b405-2a6f4435810b service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:26.278647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210f8b9-e75b-4664-b405-2a6f4435810b service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:26.278911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210f8b9-e75b-4664-b405-2a6f4435810b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:26.280547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8210f8b9-e75b-4664-b405-2a6f4435810b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:26.370556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:07:26.400274 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210f8b9-e75b-4664-b405-2a6f4435810b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:07:26.401768 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210f8b9-e75b-4664-b405-2a6f4435810b service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:07:26.420379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 755/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:07:28.532645 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbb2c6-ff0b-43f3-8347-206dc165fa6d None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:28.533532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbb2c6-ff0b-43f3-8347-206dc165fa6d None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:28.533532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbb2c6-ff0b-43f3-8347-206dc165fa6d None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:28.535799 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 736/1491] 10.222.0.17 () {54 vars in 927 bytes} [Tue Aug 23 07:07:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:07:28.542073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cc732-d22c-47da-b634-3aff2c19d046 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:28.542755 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cc732-d22c-47da-b634-3aff2c19d046 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:28.543012 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cc732-d22c-47da-b634-3aff2c19d046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:28.629779 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-270cc732-d22c-47da-b634-3aff2c19d046 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:07:28.631009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-270cc732-d22c-47da-b634-3aff2c19d046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:28.754130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 756/1492] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Aug 23 07:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:07:29.039475 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbb2c6-ff0b-43f3-8347-206dc165fa6d None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:07:29.040961 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e2dbb2c6-ff0b-43f3-8347-206dc165fa6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:29.156001 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbb2c6-ff0b-43f3-8347-206dc165fa6d None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:07:29.243142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410b9d4-6a7b-4d01-bfa2-f460115ab110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:07:29.243843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410b9d4-6a7b-4d01-bfa2-f460115ab110 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:07:29.244114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410b9d4-6a7b-4d01-bfa2-f460115ab110 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:07:29.244378 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410b9d4-6a7b-4d01-bfa2-f460115ab110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:07:29.246060 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c410b9d4-6a7b-4d01-bfa2-f460115ab110 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:07:29.254721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:07:29.373768 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c410b9d4-6a7b-4d01-bfa2-f460115ab110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:07:29.375212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c410b9d4-6a7b-4d01-bfa2-f460115ab110 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:07:29.393881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 737/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:07:38.752165 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=82 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:07:39.391539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=50 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:08:48.245473 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496cc79-11f0-4e9f-ade9-54d651d889ea None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:08:48.246080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496cc79-11f0-4e9f-ade9-54d651d889ea None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:08:48.246199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496cc79-11f0-4e9f-ade9-54d651d889ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:08:48.327666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-b496cc79-11f0-4e9f-ade9-54d651d889ea None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:08:48.329230 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b496cc79-11f0-4e9f-ade9-54d651d889ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:08:48.432047 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 757/1494] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:08:48.440482 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e6599-5a71-49a1-b705-e9328ff54f24 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:08:48.441000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e6599-5a71-49a1-b705-e9328ff54f24 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:08:48.441000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e6599-5a71-49a1-b705-e9328ff54f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:08:48.526739 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-959e6599-5a71-49a1-b705-e9328ff54f24 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:08:48.528165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-959e6599-5a71-49a1-b705-e9328ff54f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:08:48.628262 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 738/1495] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:08:48.633667 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496cc79-11f0-4e9f-ade9-54d651d889ea None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:08:48.635528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b496cc79-11f0-4e9f-ade9-54d651d889ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:08:48.652467 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496cc79-11f0-4e9f-ade9-54d651d889ea None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:08:48.719355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:08:48.754100 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db156f7-09a4-4def-8c20-47ae0a0e3bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:08:48.754699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db156f7-09a4-4def-8c20-47ae0a0e3bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:08:48.754971 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db156f7-09a4-4def-8c20-47ae0a0e3bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:08:48.755242 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db156f7-09a4-4def-8c20-47ae0a0e3bd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:08:48.755921 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db156f7-09a4-4def-8c20-47ae0a0e3bd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:08:48.756221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db156f7-09a4-4def-8c20-47ae0a0e3bd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:08:48.757236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: WARNING py.warnings [None req-5db156f7-09a4-4def-8c20-47ae0a0e3bd4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 07:08:48.757236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: warnings.warn(msg) Aug 23 07:08:48.757236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: Aug 23 07:08:48.758122 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db156f7-09a4-4def-8c20-47ae0a0e3bd4 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:08:48.769968 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 758/1496] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:08:48.778594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e6599-5a71-49a1-b705-e9328ff54f24 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:08:48.788030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e6599-5a71-49a1-b705-e9328ff54f24 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:08:48.856411 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:08:48.896045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cda3d2-ea55-4b76-bfc7-fd9df685e492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:08:48.896695 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cda3d2-ea55-4b76-bfc7-fd9df685e492 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:08:48.896961 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cda3d2-ea55-4b76-bfc7-fd9df685e492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:08:48.897224 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cda3d2-ea55-4b76-bfc7-fd9df685e492 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:08:48.897843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cda3d2-ea55-4b76-bfc7-fd9df685e492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:08:48.899632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: WARNING py.warnings [None req-d0cda3d2-ea55-4b76-bfc7-fd9df685e492 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 07:08:48.899632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: warnings.warn(msg) Aug 23 07:08:48.899632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: Aug 23 07:08:48.900822 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cda3d2-ea55-4b76-bfc7-fd9df685e492 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:08:48.960927 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 739/1497] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:08:48 2022] POST /identity/v3/projects => generated 377 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:08:48.965457 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db156f7-09a4-4def-8c20-47ae0a0e3bd4 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:08:48.966979 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-5db156f7-09a4-4def-8c20-47ae0a0e3bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:08:49.002476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db156f7-09a4-4def-8c20-47ae0a0e3bd4 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:08:49.110029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a519be-35ee-440b-b55f-0742f385fe57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:08:49.110643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a519be-35ee-440b-b55f-0742f385fe57 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:08:49.110916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a519be-35ee-440b-b55f-0742f385fe57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:08:49.111182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a519be-35ee-440b-b55f-0742f385fe57 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:08:49.111854 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a519be-35ee-440b-b55f-0742f385fe57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:08:49.112907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: WARNING py.warnings [None req-a7a519be-35ee-440b-b55f-0742f385fe57 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 07:08:49.112907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: warnings.warn(msg) Aug 23 07:08:49.112907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: Aug 23 07:08:49.114104 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a519be-35ee-440b-b55f-0742f385fe57 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:08:49.159249 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 759/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:08:48 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:08:49.163523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cda3d2-ea55-4b76-bfc7-fd9df685e492 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:08:49.165051 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d0cda3d2-ea55-4b76-bfc7-fd9df685e492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:08:49.229313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cda3d2-ea55-4b76-bfc7-fd9df685e492 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:08:49.320457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:08:49.334157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cb5ea-f587-47bd-9320-44e8b9c17e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:08:49.334765 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cb5ea-f587-47bd-9320-44e8b9c17e41 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:08:49.335036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cb5ea-f587-47bd-9320-44e8b9c17e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:08:49.335304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cb5ea-f587-47bd-9320-44e8b9c17e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:08:49.335975 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cb5ea-f587-47bd-9320-44e8b9c17e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:08:49.336995 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: WARNING py.warnings [None req-9c8cb5ea-f587-47bd-9320-44e8b9c17e41 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 07:08:49.336995 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: warnings.warn(msg) Aug 23 07:08:49.336995 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: Aug 23 07:08:49.337904 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cb5ea-f587-47bd-9320-44e8b9c17e41 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:08:49.350393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 740/1499] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:08:49.354671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a519be-35ee-440b-b55f-0742f385fe57 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:08:49.356376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a7a519be-35ee-440b-b55f-0742f385fe57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:08:49.368170 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a519be-35ee-440b-b55f-0742f385fe57 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:08:49.399029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:08:49.477042 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f151d2-40a1-4abc-a9d2-8d9d634e1d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:08:49.477655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f151d2-40a1-4abc-a9d2-8d9d634e1d11 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:08:49.477924 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f151d2-40a1-4abc-a9d2-8d9d634e1d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:08:49.478196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f151d2-40a1-4abc-a9d2-8d9d634e1d11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:08:49.478840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f151d2-40a1-4abc-a9d2-8d9d634e1d11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:08:49.479908 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: WARNING py.warnings [None req-72f151d2-40a1-4abc-a9d2-8d9d634e1d11 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 07:08:49.479908 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: warnings.warn(msg) Aug 23 07:08:49.479908 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: Aug 23 07:08:49.480838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f151d2-40a1-4abc-a9d2-8d9d634e1d11 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:08:49.493370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 760/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:08:49.497704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cb5ea-f587-47bd-9320-44e8b9c17e41 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:08:49.499246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9c8cb5ea-f587-47bd-9320-44e8b9c17e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:08:49.533803 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cb5ea-f587-47bd-9320-44e8b9c17e41 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:08:49.642844 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67721793-4047-450c-a234-30ab2d0fd90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:08:49.643533 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67721793-4047-450c-a234-30ab2d0fd90d admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:08:49.643807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67721793-4047-450c-a234-30ab2d0fd90d admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:08:49.644130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67721793-4047-450c-a234-30ab2d0fd90d admin admin] PATH_INFO: `/v3/projects/9f5e9ab9ab304057a97763fb816f84e6/users/81be05bed41e4ac388a3f6fefe944e7e/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:08:49.668939 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67721793-4047-450c-a234-30ab2d0fd90d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f5e9ab9ab304057a97763fb816f84e6, user_id=81be05bed41e4ac388a3f6fefe944e7e, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:08:49.670009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: WARNING py.warnings [None req-67721793-4047-450c-a234-30ab2d0fd90d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 07:08:49.670009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: warnings.warn(msg) Aug 23 07:08:49.670009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: Aug 23 07:08:49.672036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67721793-4047-450c-a234-30ab2d0fd90d admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:08:49.700208 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 741/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:08:49 2022] PUT /identity/v3/projects/9f5e9ab9ab304057a97763fb816f84e6/users/81be05bed41e4ac388a3f6fefe944e7e/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:08:49.712921 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b794bb6-2501-4659-9d4b-62a6c3767f9a None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:08:49.713200 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b794bb6-2501-4659-9d4b-62a6c3767f9a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:08:49.713471 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b794bb6-2501-4659-9d4b-62a6c3767f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:08:49.772302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-1b794bb6-2501-4659-9d4b-62a6c3767f9a None None] MFA Rules not processed for user `81be05bed41e4ac388a3f6fefe944e7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:08:49.773511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-1b794bb6-2501-4659-9d4b-62a6c3767f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:08:49.862054 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 761/1502] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:08:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:08:49.875581 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809db3c6-2e5f-4835-bb87-8601a732d419 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:08:49.875868 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809db3c6-2e5f-4835-bb87-8601a732d419 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:08:49.876137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809db3c6-2e5f-4835-bb87-8601a732d419 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:08:49.877492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 742/1503] 10.222.0.17 () {54 vars in 964 bytes} [Tue Aug 23 07:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:08:49.882435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3e583-72d6-4bb3-b45e-95054c806484 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:08:49.882720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3e583-72d6-4bb3-b45e-95054c806484 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:08:49.882993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3e583-72d6-4bb3-b45e-95054c806484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:08:49.916369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:08:49.963236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-a7d3e583-72d6-4bb3-b45e-95054c806484 None None] MFA Rules not processed for user `a26f09e892a74d9eab97e73cce9d062a`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:08:49.964597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a7d3e583-72d6-4bb3-b45e-95054c806484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:08:50.049780 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 762/1504] 10.222.0.17 () {58 vars in 1061 bytes} [Tue Aug 23 07:08:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:08:50.061106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809db3c6-2e5f-4835-bb87-8601a732d419 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:08:50.062642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-809db3c6-2e5f-4835-bb87-8601a732d419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:08:50.123128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:08:50.181959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809db3c6-2e5f-4835-bb87-8601a732d419 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:08:50.275953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337a162-8b8a-49bb-aa97-75a93f9d7f24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:08:50.276590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337a162-8b8a-49bb-aa97-75a93f9d7f24 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:08:50.276844 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337a162-8b8a-49bb-aa97-75a93f9d7f24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:08:50.277783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337a162-8b8a-49bb-aa97-75a93f9d7f24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:08:50.279400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2337a162-8b8a-49bb-aa97-75a93f9d7f24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:08:50.400211 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337a162-8b8a-49bb-aa97-75a93f9d7f24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:08:50.401659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337a162-8b8a-49bb-aa97-75a93f9d7f24 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:08:50.408360 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:08:50.426535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 743/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:08:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:08:50.473960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4eabb-896a-4ae2-a572-371dbffdf88a None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:08:50.474369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4eabb-896a-4ae2-a572-371dbffdf88a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:08:50.474510 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4eabb-896a-4ae2-a572-371dbffdf88a None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:08:50.475848 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 763/1506] 10.222.0.17 () {52 vars in 830 bytes} [Tue Aug 23 07:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:08:50.479595 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337a162-8b8a-49bb-aa97-75a93f9d7f24 service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:08:50.482756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2337a162-8b8a-49bb-aa97-75a93f9d7f24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:08:50.590775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337a162-8b8a-49bb-aa97-75a93f9d7f24 service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:08:50.697259 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28864866-2bc8-423c-9117-ae5648e35440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:08:50.697826 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28864866-2bc8-423c-9117-ae5648e35440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:08:50.698104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28864866-2bc8-423c-9117-ae5648e35440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:08:50.698375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28864866-2bc8-423c-9117-ae5648e35440 admin admin] PATH_INFO: `/v3/users/81be05bed41e4ac388a3f6fefe944e7e` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:08:50.712233 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28864866-2bc8-423c-9117-ae5648e35440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81be05bed41e4ac388a3f6fefe944e7e)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:08:50.713283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: WARNING py.warnings [None req-28864866-2bc8-423c-9117-ae5648e35440 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 07:08:50.713283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: warnings.warn(msg) Aug 23 07:08:50.713283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: Aug 23 07:08:50.714584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28864866-2bc8-423c-9117-ae5648e35440 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:08:50.724512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:08:50.789623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-28864866-2bc8-423c-9117-ae5648e35440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81be05bed41e4ac388a3f6fefe944e7e'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:08:50.790158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-28864866-2bc8-423c-9117-ae5648e35440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81be05bed41e4ac388a3f6fefe944e7e'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:08:50.790570 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-28864866-2bc8-423c-9117-ae5648e35440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81be05bed41e4ac388a3f6fefe944e7e'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:08:50.804975 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-28864866-2bc8-423c-9117-ae5648e35440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81be05bed41e4ac388a3f6fefe944e7e'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:08:50.828906 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-28864866-2bc8-423c-9117-ae5648e35440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81be05bed41e4ac388a3f6fefe944e7e'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:08:50.845393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 744/1507] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:08:50 2022] DELETE /identity/v3/users/81be05bed41e4ac388a3f6fefe944e7e => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:08:50.849957 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4ac48-2391-4e15-8e07-91f0c452171b None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:08:50.850309 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4ac48-2391-4e15-8e07-91f0c452171b None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:08:50.850582 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4ac48-2391-4e15-8e07-91f0c452171b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:08:50.908147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-78a4ac48-2391-4e15-8e07-91f0c452171b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:08:50.909345 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-78a4ac48-2391-4e15-8e07-91f0c452171b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:08:50.987471 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:08:51.000311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 764/1508] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:08:51.012301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28864866-2bc8-423c-9117-ae5648e35440 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:08:51.013537 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-28864866-2bc8-423c-9117-ae5648e35440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:08:51.115345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28864866-2bc8-423c-9117-ae5648e35440 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:08:51.176526 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:08:51.197538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f12cc0-55bf-49f1-b109-a8687f882dea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:08:51.198166 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f12cc0-55bf-49f1-b109-a8687f882dea service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:08:51.198438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f12cc0-55bf-49f1-b109-a8687f882dea service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:08:51.198707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f12cc0-55bf-49f1-b109-a8687f882dea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:08:51.200390 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f4f12cc0-55bf-49f1-b109-a8687f882dea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:08:51.328823 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f12cc0-55bf-49f1-b109-a8687f882dea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:08:51.330190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f12cc0-55bf-49f1-b109-a8687f882dea service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:08:51.348903 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 745/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:08:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:08:51.715474 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4ac48-2391-4e15-8e07-91f0c452171b None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:08:51.716726 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-78a4ac48-2391-4e15-8e07-91f0c452171b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:08:51.728488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4ac48-2391-4e15-8e07-91f0c452171b None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:08:51.834711 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f6b68-1750-435d-90b5-215746258cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:08:51.835604 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f6b68-1750-435d-90b5-215746258cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:08:51.836000 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f6b68-1750-435d-90b5-215746258cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:08:51.836382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f6b68-1750-435d-90b5-215746258cfc admin admin] PATH_INFO: `/v3/projects/9f5e9ab9ab304057a97763fb816f84e6` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:08:51.852073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f6b68-1750-435d-90b5-215746258cfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f5e9ab9ab304057a97763fb816f84e6)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:08:51.853527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: WARNING py.warnings [None req-672f6b68-1750-435d-90b5-215746258cfc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 07:08:51.853527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: warnings.warn(msg) Aug 23 07:08:51.853527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: Aug 23 07:08:51.855361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f6b68-1750-435d-90b5-215746258cfc admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:08:51.919426 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-672f6b68-1750-435d-90b5-215746258cfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f5e9ab9ab304057a97763fb816f84e6', 'request_id': 'req-672f6b68-1750-435d-90b5-215746258cfc'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:08:51.925976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-672f6b68-1750-435d-90b5-215746258cfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f5e9ab9ab304057a97763fb816f84e6', 'request_id': 'req-672f6b68-1750-435d-90b5-215746258cfc'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:08:51.936132 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-672f6b68-1750-435d-90b5-215746258cfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f5e9ab9ab304057a97763fb816f84e6', 'request_id': 'req-672f6b68-1750-435d-90b5-215746258cfc'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:08:51.966794 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-672f6b68-1750-435d-90b5-215746258cfc admin admin] The token cache is being invalidate because project 9f5e9ab9ab304057a97763fb816f84e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:08:51.967075 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-672f6b68-1750-435d-90b5-215746258cfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:08:51.967618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-672f6b68-1750-435d-90b5-215746258cfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:08:51.968920 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 765/1510] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:08:51 2022] DELETE /identity/v3/projects/9f5e9ab9ab304057a97763fb816f84e6 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:09:01.346149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=62 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:09:01.964763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:10:17.308211 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0ccd0-506c-412b-bfdf-811db7916d3d None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:17.308915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0ccd0-506c-412b-bfdf-811db7916d3d None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:17.308915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0ccd0-506c-412b-bfdf-811db7916d3d None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:17.310694 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 07:10:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:10:17.317933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5c475-715c-46d3-9b95-2de0e262e283 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:17.318391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5c475-715c-46d3-9b95-2de0e262e283 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:17.318497 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5c475-715c-46d3-9b95-2de0e262e283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:17.408623 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-37c5c475-715c-46d3-9b95-2de0e262e283 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:10:17.409847 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-37c5c475-715c-46d3-9b95-2de0e262e283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:10:17.522645 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 07:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:10:17.681893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0ccd0-506c-412b-bfdf-811db7916d3d None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:10:17.684765 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b4b0ccd0-506c-412b-bfdf-811db7916d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:10:17.836180 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0ccd0-506c-412b-bfdf-811db7916d3d None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:10:17.842351 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0abdd6-4db8-43c0-9d5e-dad4888b004c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:10:17.842962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0abdd6-4db8-43c0-9d5e-dad4888b004c service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:17.843234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0abdd6-4db8-43c0-9d5e-dad4888b004c service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:17.843541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0abdd6-4db8-43c0-9d5e-dad4888b004c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:17.852414 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0abdd6-4db8-43c0-9d5e-dad4888b004c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:10:17.854013 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0abdd6-4db8-43c0-9d5e-dad4888b004c service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:10:17.864101 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 747/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:10:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:10:18.691800 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d67eb-bcd1-4d18-8a28-476e0a09544a None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:18.692321 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d67eb-bcd1-4d18-8a28-476e0a09544a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:18.692321 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d67eb-bcd1-4d18-8a28-476e0a09544a None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:18.693706 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 07:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:10:18.698984 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c8e67-efcc-4be7-b1ae-99d90224ac1d None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:18.699397 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c8e67-efcc-4be7-b1ae-99d90224ac1d None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:18.699597 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c8e67-efcc-4be7-b1ae-99d90224ac1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:18.720354 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-ac8c8e67-efcc-4be7-b1ae-99d90224ac1d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:10:18.721555 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ac8c8e67-efcc-4be7-b1ae-99d90224ac1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:10:18.729869 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 07:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:10:18.885406 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d67eb-bcd1-4d18-8a28-476e0a09544a None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:10:18.902219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d67eb-bcd1-4d18-8a28-476e0a09544a None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:10:18.911024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475a369-c387-4e1d-aa70-5802de2cd22e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:10:18.911674 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475a369-c387-4e1d-aa70-5802de2cd22e service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:18.911970 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475a369-c387-4e1d-aa70-5802de2cd22e service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:18.912240 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475a369-c387-4e1d-aa70-5802de2cd22e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:18.923513 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475a369-c387-4e1d-aa70-5802de2cd22e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:10:18.925117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475a369-c387-4e1d-aa70-5802de2cd22e service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:10:18.935070 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 768/1516] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:10:18.942253 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a9648-9181-4468-9058-605a1631466a None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:18.942539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a9648-9181-4468-9058-605a1631466a None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:18.942986 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a9648-9181-4468-9058-605a1631466a None None] PATH_INFO: `/v3` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:18.944228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 749/1517] 10.222.0.20 () {54 vars in 925 bytes} [Tue Aug 23 07:10:18 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:10:18.950348 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a50a1d-a346-4051-ab29-2425612402bb None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:18.950629 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a50a1d-a346-4051-ab29-2425612402bb None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:18.950870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a50a1d-a346-4051-ab29-2425612402bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:19.020160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-d0a50a1d-a346-4051-ab29-2425612402bb None None] MFA Rules not processed for user `a9b0f54c988e43bda474f978a69fa4fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:10:19.021467 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d0a50a1d-a346-4051-ab29-2425612402bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:10:19.048116 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:10:19.105779 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 769/1518] 10.222.0.20 () {58 vars in 1015 bytes} [Tue Aug 23 07:10:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:10:22.863123 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec78bf-2441-4e45-9103-d08c67867262 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:22.863704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec78bf-2441-4e45-9103-d08c67867262 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:22.863704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec78bf-2441-4e45-9103-d08c67867262 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:22.864957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 750/1519] 10.222.0.17 () {54 vars in 961 bytes} [Tue Aug 23 07:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:10:22.870089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7942a-1b53-4fce-a00a-c8708405d5ad None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:22.870600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7942a-1b53-4fce-a00a-c8708405d5ad None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:22.870833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7942a-1b53-4fce-a00a-c8708405d5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:22.897114 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-dae7942a-1b53-4fce-a00a-c8708405d5ad None None] MFA Rules not processed for user `a9b0f54c988e43bda474f978a69fa4fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:10:22.898351 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-dae7942a-1b53-4fce-a00a-c8708405d5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:10:22.908415 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 770/1520] 10.222.0.17 () {58 vars in 1058 bytes} [Tue Aug 23 07:10:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:10:22.916201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec78bf-2441-4e45-9103-d08c67867262 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:10:22.930769 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec78bf-2441-4e45-9103-d08c67867262 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:10:22.936894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54920571-2f67-41d1-81d2-94cce9812f46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9b0f54c988e43bda474f978a69fa4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:10:22.937480 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54920571-2f67-41d1-81d2-94cce9812f46 service placement] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:22.937780 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54920571-2f67-41d1-81d2-94cce9812f46 service placement] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:22.938039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54920571-2f67-41d1-81d2-94cce9812f46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:22.948127 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54920571-2f67-41d1-81d2-94cce9812f46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:10:22.949836 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54920571-2f67-41d1-81d2-94cce9812f46 service placement] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:10:22.960605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 751/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:10:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:10:30.216453 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186c168-9145-4b33-a78a-d176bd90a1e5 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:30.217154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186c168-9145-4b33-a78a-d176bd90a1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:30.217154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186c168-9145-4b33-a78a-d176bd90a1e5 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:30.219472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 07:10:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:10:30.224490 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cc527-1f73-480f-9dc8-b54c1a4d3770 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:30.224945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cc527-1f73-480f-9dc8-b54c1a4d3770 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:30.225057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cc527-1f73-480f-9dc8-b54c1a4d3770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:30.252274 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-4c8cc527-1f73-480f-9dc8-b54c1a4d3770 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:10:30.254096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-4c8cc527-1f73-480f-9dc8-b54c1a4d3770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:10:30.266421 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 07:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:10:30.423643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186c168-9145-4b33-a78a-d176bd90a1e5 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:10:30.443047 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186c168-9145-4b33-a78a-d176bd90a1e5 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:10:30.449633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb77b9-04be-4eae-aee1-556173f8187e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:10:30.450269 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb77b9-04be-4eae-aee1-556173f8187e service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:30.450568 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb77b9-04be-4eae-aee1-556173f8187e service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:30.450836 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb77b9-04be-4eae-aee1-556173f8187e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:30.459365 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb77b9-04be-4eae-aee1-556173f8187e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:10:30.460915 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb77b9-04be-4eae-aee1-556173f8187e service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:10:30.470510 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 772/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:10:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:10:31.296604 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae11fff-899c-48ac-9d8d-0ea66f55190f None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:31.297479 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae11fff-899c-48ac-9d8d-0ea66f55190f None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:31.297479 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae11fff-899c-48ac-9d8d-0ea66f55190f None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:31.299398 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 07:10:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:10:31.306779 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a510f-4892-47c5-9ac5-7733265e279e None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:31.307217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a510f-4892-47c5-9ac5-7733265e279e None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:31.307325 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a510f-4892-47c5-9ac5-7733265e279e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:31.332044 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-a45a510f-4892-47c5-9ac5-7733265e279e None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:10:31.333250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a45a510f-4892-47c5-9ac5-7733265e279e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:10:31.342352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 07:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:10:31.494886 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae11fff-899c-48ac-9d8d-0ea66f55190f None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:10:31.516095 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae11fff-899c-48ac-9d8d-0ea66f55190f None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:10:31.522796 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921bb91-a6a9-4d8d-8755-24513303cc2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:10:31.527594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921bb91-a6a9-4d8d-8755-24513303cc2f service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:31.527594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921bb91-a6a9-4d8d-8755-24513303cc2f service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:31.527594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921bb91-a6a9-4d8d-8755-24513303cc2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:31.534737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921bb91-a6a9-4d8d-8755-24513303cc2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:10:31.536365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921bb91-a6a9-4d8d-8755-24513303cc2f service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:10:31.546470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 754/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:10:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:10:35.256245 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c669d-f99d-4f37-9ed2-44f772a401d5 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:35.257087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c669d-f99d-4f37-9ed2-44f772a401d5 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:35.257087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c669d-f99d-4f37-9ed2-44f772a401d5 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:35.259187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 07:10:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:10:35.265649 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fcb3a-36b7-4841-9560-98a26c32d85f None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:35.266126 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fcb3a-36b7-4841-9560-98a26c32d85f None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:35.266244 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fcb3a-36b7-4841-9560-98a26c32d85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:35.290813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-5c1fcb3a-36b7-4841-9560-98a26c32d85f None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:10:35.292070 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-5c1fcb3a-36b7-4841-9560-98a26c32d85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:10:35.300364 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 07:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:10:35.588930 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c669d-f99d-4f37-9ed2-44f772a401d5 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:10:35.591466 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-2e3c669d-f99d-4f37-9ed2-44f772a401d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:10:35.678684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c669d-f99d-4f37-9ed2-44f772a401d5 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:10:35.683287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67552e-fbb4-429f-8b63-21a0d82f1d12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:10:35.683941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67552e-fbb4-429f-8b63-21a0d82f1d12 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:35.684215 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67552e-fbb4-429f-8b63-21a0d82f1d12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:35.684481 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67552e-fbb4-429f-8b63-21a0d82f1d12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:35.694212 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67552e-fbb4-429f-8b63-21a0d82f1d12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:10:35.695680 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67552e-fbb4-429f-8b63-21a0d82f1d12 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:10:35.705824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 775/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:10:37.587134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dced44f-1820-43f8-99fe-a172fbec5de1 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:37.587782 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dced44f-1820-43f8-99fe-a172fbec5de1 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:37.587782 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dced44f-1820-43f8-99fe-a172fbec5de1 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:37.589192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 07:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:10:37.595769 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022328b6-c8a5-4dec-a9da-9a83287957d5 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:37.596231 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022328b6-c8a5-4dec-a9da-9a83287957d5 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:37.596337 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022328b6-c8a5-4dec-a9da-9a83287957d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:37.626298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-022328b6-c8a5-4dec-a9da-9a83287957d5 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:10:37.627535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-022328b6-c8a5-4dec-a9da-9a83287957d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:10:37.636564 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 07:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:10:37.932331 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dced44f-1820-43f8-99fe-a172fbec5de1 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:10:37.949811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dced44f-1820-43f8-99fe-a172fbec5de1 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:10:37.954409 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bc698-2235-4822-9894-7efb169f6f06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:10:37.955330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bc698-2235-4822-9894-7efb169f6f06 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:37.955643 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bc698-2235-4822-9894-7efb169f6f06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:37.955917 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bc698-2235-4822-9894-7efb169f6f06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:37.966067 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6bc698-2235-4822-9894-7efb169f6f06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:10:37.967566 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6bc698-2235-4822-9894-7efb169f6f06 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:10:37.977334 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 757/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:10:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:10:41.371551 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a61831-e21b-428e-807f-c502065da8f8 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:41.372842 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a61831-e21b-428e-807f-c502065da8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:41.372842 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a61831-e21b-428e-807f-c502065da8f8 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:41.374723 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 07:10:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:10:41.383812 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5edef-19e9-432c-b761-434fc75d3bad None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:41.384255 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5edef-19e9-432c-b761-434fc75d3bad None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:41.384395 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5edef-19e9-432c-b761-434fc75d3bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:41.396109 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=81 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:10:41.412281 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-ada5edef-19e9-432c-b761-434fc75d3bad None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:10:41.413694 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ada5edef-19e9-432c-b761-434fc75d3bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:10:41.422941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 07:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:10:41.984424 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a61831-e21b-428e-807f-c502065da8f8 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:10:41.987370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-03a61831-e21b-428e-807f-c502065da8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:10:42.068162 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:10:42.083173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a61831-e21b-428e-807f-c502065da8f8 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:10:42.087692 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ed17d-fdce-4922-9042-45f45623d328 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:10:42.088487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ed17d-fdce-4922-9042-45f45623d328 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:42.088845 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ed17d-fdce-4922-9042-45f45623d328 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:42.089243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ed17d-fdce-4922-9042-45f45623d328 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:42.101476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ed17d-fdce-4922-9042-45f45623d328 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:10:42.103119 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ed17d-fdce-4922-9042-45f45623d328 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:10:42.114050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 778/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:10:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:10:44.044827 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269857c-e419-4c96-894a-3cf8e8b73bf5 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:44.045427 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269857c-e419-4c96-894a-3cf8e8b73bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:44.045427 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269857c-e419-4c96-894a-3cf8e8b73bf5 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:44.046659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 07:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:10:44.053174 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4d925-67d6-4d4f-8dd7-21d664e5f7cf None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:44.053900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4d925-67d6-4d4f-8dd7-21d664e5f7cf None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:44.054088 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4d925-67d6-4d4f-8dd7-21d664e5f7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:44.082148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-f6f4d925-67d6-4d4f-8dd7-21d664e5f7cf None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:10:44.083358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-f6f4d925-67d6-4d4f-8dd7-21d664e5f7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:10:44.092442 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 07:10:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:10:44.631713 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269857c-e419-4c96-894a-3cf8e8b73bf5 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:10:44.633586 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e269857c-e419-4c96-894a-3cf8e8b73bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:10:44.649190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269857c-e419-4c96-894a-3cf8e8b73bf5 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:10:44.653230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04861af8-35d6-4241-81a6-42824334e20c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:10:44.653842 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04861af8-35d6-4241-81a6-42824334e20c service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:10:44.654222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04861af8-35d6-4241-81a6-42824334e20c service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:10:44.654485 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04861af8-35d6-4241-81a6-42824334e20c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:10:44.664371 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04861af8-35d6-4241-81a6-42824334e20c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:10:44.665810 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04861af8-35d6-4241-81a6-42824334e20c service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:10:44.675168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 760/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:10:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:10:54.077190 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=11 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:10:54.671148 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=8 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:07.441437 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9db40-0757-4e96-8bbb-ad0c808f809e None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:07.468287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9db40-0757-4e96-8bbb-ad0c808f809e None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:07.468287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9db40-0757-4e96-8bbb-ad0c808f809e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:07.468287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-f1f9db40-0757-4e96-8bbb-ad0c808f809e None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:07.468287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-f1f9db40-0757-4e96-8bbb-ad0c808f809e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:07.479787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 780/1540] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:07.490827 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8772a-b312-4eaa-80df-a739cea3f67e None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:07.491435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8772a-b312-4eaa-80df-a739cea3f67e None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:07.491435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8772a-b312-4eaa-80df-a739cea3f67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:07.516567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-10e8772a-b312-4eaa-80df-a739cea3f67e None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:07.517810 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-10e8772a-b312-4eaa-80df-a739cea3f67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:07.531234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 761/1541] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:07.536558 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9db40-0757-4e96-8bbb-ad0c808f809e None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:07.546950 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9db40-0757-4e96-8bbb-ad0c808f809e None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:07.555446 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80b97b-f14e-4b2f-8bda-3ed75fec8130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:07.556074 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80b97b-f14e-4b2f-8bda-3ed75fec8130 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:07.556430 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80b97b-f14e-4b2f-8bda-3ed75fec8130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:07.557024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80b97b-f14e-4b2f-8bda-3ed75fec8130 admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:07.557381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80b97b-f14e-4b2f-8bda-3ed75fec8130 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:12:07.557687 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80b97b-f14e-4b2f-8bda-3ed75fec8130 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:07.559483 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80b97b-f14e-4b2f-8bda-3ed75fec8130 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:07.582023 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 781/1542] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:12:07.590519 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8772a-b312-4eaa-80df-a739cea3f67e None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:07.604822 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8772a-b312-4eaa-80df-a739cea3f67e None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:07.609539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ee336-19af-4bcc-bc00-fe70bfc6c883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:07.610161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ee336-19af-4bcc-bc00-fe70bfc6c883 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:07.610440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ee336-19af-4bcc-bc00-fe70bfc6c883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:07.610711 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ee336-19af-4bcc-bc00-fe70bfc6c883 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:07.611345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ee336-19af-4bcc-bc00-fe70bfc6c883 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:07.613453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ee336-19af-4bcc-bc00-fe70bfc6c883 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:07.676478 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 762/1543] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:07 2022] POST /identity/v3/projects => generated 391 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:07.684023 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80b97b-f14e-4b2f-8bda-3ed75fec8130 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:07.702357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80b97b-f14e-4b2f-8bda-3ed75fec8130 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:07.785984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe77ad-1eb7-434c-8de4-2e4242cda018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:07.786875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe77ad-1eb7-434c-8de4-2e4242cda018 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:07.787010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe77ad-1eb7-434c-8de4-2e4242cda018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:07.787280 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe77ad-1eb7-434c-8de4-2e4242cda018 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:07.788140 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe77ad-1eb7-434c-8de4-2e4242cda018 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:07.791429 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe77ad-1eb7-434c-8de4-2e4242cda018 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:07.826199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 782/1544] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:07 2022] POST /identity/v3/users => generated 296 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:07.831074 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ee336-19af-4bcc-bc00-fe70bfc6c883 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:07.850120 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ee336-19af-4bcc-bc00-fe70bfc6c883 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:07.855133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff7d1e-e0bf-477c-b635-7ec687d38131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:07.855785 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff7d1e-e0bf-477c-b635-7ec687d38131 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:07.856067 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff7d1e-e0bf-477c-b635-7ec687d38131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:07.856780 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff7d1e-e0bf-477c-b635-7ec687d38131 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:07.856981 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff7d1e-e0bf-477c-b635-7ec687d38131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:07.858667 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff7d1e-e0bf-477c-b635-7ec687d38131 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:07.878543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 763/1545] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:07.879199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab7377-6195-4c68-bc40-0a97c9456273 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:07.879901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab7377-6195-4c68-bc40-0a97c9456273 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:07.880402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab7377-6195-4c68-bc40-0a97c9456273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:07.882876 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff7d1e-e0bf-477c-b635-7ec687d38131 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:07.896348 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff7d1e-e0bf-477c-b635-7ec687d38131 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:07.901451 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd66a5-19c5-4383-9471-f382838dac36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:07.902075 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd66a5-19c5-4383-9471-f382838dac36 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:07.902355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd66a5-19c5-4383-9471-f382838dac36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:07.902631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd66a5-19c5-4383-9471-f382838dac36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:07.903305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd66a5-19c5-4383-9471-f382838dac36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:07.904996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd66a5-19c5-4383-9471-f382838dac36 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:07.922513 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-fbab7377-6195-4c68-bc40-0a97c9456273 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:07.924329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-fbab7377-6195-4c68-bc40-0a97c9456273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:07.927876 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 764/1546] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:12:07.929209 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd66a5-19c5-4383-9471-f382838dac36 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:07.943366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd66a5-19c5-4383-9471-f382838dac36 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:07.950625 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 783/1547] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 23 07:12:07.960131 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becb5ed-49bb-483c-9b83-a9664aac9221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:07.960750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becb5ed-49bb-483c-9b83-a9664aac9221 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:07.961030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becb5ed-49bb-483c-9b83-a9664aac9221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:07.961305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becb5ed-49bb-483c-9b83-a9664aac9221 admin admin] PATH_INFO: `/v3/projects/135367be6e3f4b3d965ee0d99fdca524/users/64d54cb95b264c18b45d90822507fae1/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:07.962043 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dc95d-4977-4fd3-8814-212973fbff80 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:07.962956 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dc95d-4977-4fd3-8814-212973fbff80 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:07.963573 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dc95d-4977-4fd3-8814-212973fbff80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:07.992885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2becb5ed-49bb-483c-9b83-a9664aac9221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=135367be6e3f4b3d965ee0d99fdca524, user_id=64d54cb95b264c18b45d90822507fae1, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:07.995764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2becb5ed-49bb-483c-9b83-a9664aac9221 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:08.003099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-1d2dc95d-4977-4fd3-8814-212973fbff80 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:08.005282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-1d2dc95d-4977-4fd3-8814-212973fbff80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:08.020207 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 765/1548] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:07 2022] PUT /identity/v3/projects/135367be6e3f4b3d965ee0d99fdca524/users/64d54cb95b264c18b45d90822507fae1/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:08.025766 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c8af1-8da6-4ca2-aec8-f3b599d9ec17 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.026082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c8af1-8da6-4ca2-aec8-f3b599d9ec17 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.026356 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c8af1-8da6-4ca2-aec8-f3b599d9ec17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.069193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-ec0c8af1-8da6-4ca2-aec8-f3b599d9ec17 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:08.070526 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ec0c8af1-8da6-4ca2-aec8-f3b599d9ec17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:08.130547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 784/1549] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:08.136101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dc95d-4977-4fd3-8814-212973fbff80 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:08.146250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dc95d-4977-4fd3-8814-212973fbff80 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:08.162527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46ff22-1eeb-4c0b-8b85-61fb03c53ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:08.163628 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46ff22-1eeb-4c0b-8b85-61fb03c53ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.164138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46ff22-1eeb-4c0b-8b85-61fb03c53ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.164650 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46ff22-1eeb-4c0b-8b85-61fb03c53ed2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.165530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46ff22-1eeb-4c0b-8b85-61fb03c53ed2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:12:08.166193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46ff22-1eeb-4c0b-8b85-61fb03c53ed2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:08.168117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46ff22-1eeb-4c0b-8b85-61fb03c53ed2 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:08.191530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 785/1550] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:12:08.195153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46ff22-1eeb-4c0b-8b85-61fb03c53ed2 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:08.202333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 766/1551] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:08.214618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46ff22-1eeb-4c0b-8b85-61fb03c53ed2 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:08.217463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c8af1-8da6-4ca2-aec8-f3b599d9ec17 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:08.223434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc61bf3-87cc-4520-8283-d7a517cf8da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:08.223434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc61bf3-87cc-4520-8283-d7a517cf8da2 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.223434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc61bf3-87cc-4520-8283-d7a517cf8da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.223434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc61bf3-87cc-4520-8283-d7a517cf8da2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.223884 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc61bf3-87cc-4520-8283-d7a517cf8da2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:08.224911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: WARNING py.warnings [None req-fbc61bf3-87cc-4520-8283-d7a517cf8da2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 07:12:08.224911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: warnings.warn(msg) Aug 23 07:12:08.224911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: Aug 23 07:12:08.226355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc61bf3-87cc-4520-8283-d7a517cf8da2 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:08.227343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ec0c8af1-8da6-4ca2-aec8-f3b599d9ec17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:08.286828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 786/1552] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:08 2022] POST /identity/v3/projects => generated 379 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:12:08.288279 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94752e-3981-4e27-807d-f0a6f7b01051 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.288565 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94752e-3981-4e27-807d-f0a6f7b01051 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.288838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94752e-3981-4e27-807d-f0a6f7b01051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.328627 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-3f94752e-3981-4e27-807d-f0a6f7b01051 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:08.330827 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3f94752e-3981-4e27-807d-f0a6f7b01051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:08.362748 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:08.365013 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c8af1-8da6-4ca2-aec8-f3b599d9ec17 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:08.425844 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:08.434280 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 787/1553] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:08.435127 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94752e-3981-4e27-807d-f0a6f7b01051 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:08.453724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94752e-3981-4e27-807d-f0a6f7b01051 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:08.461288 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853a025-dbc3-47d2-a6bd-5c23e2f77d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:08.461902 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853a025-dbc3-47d2-a6bd-5c23e2f77d57 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.462180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853a025-dbc3-47d2-a6bd-5c23e2f77d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.462446 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853a025-dbc3-47d2-a6bd-5c23e2f77d57 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.463117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853a025-dbc3-47d2-a6bd-5c23e2f77d57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:08.464748 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2440b-4a76-4151-b1f1-9c8ace691450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:08.465313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2440b-4a76-4151-b1f1-9c8ace691450 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.465428 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853a025-dbc3-47d2-a6bd-5c23e2f77d57 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:08.466657 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2440b-4a76-4151-b1f1-9c8ace691450 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.466943 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2440b-4a76-4151-b1f1-9c8ace691450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.477064 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2440b-4a76-4151-b1f1-9c8ace691450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:08.479270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2440b-4a76-4151-b1f1-9c8ace691450 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:08.491335 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 788/1554] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:08 2022] POST /identity/v3/users => generated 290 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:08.492678 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c33e92-5047-42d6-a78f-5cd9f2a576f9 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.492962 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c33e92-5047-42d6-a78f-5cd9f2a576f9 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.493233 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c33e92-5047-42d6-a78f-5cd9f2a576f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.499722 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 767/1555] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:12:08.502678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf17e8-7dc9-4000-a744-92c98c97f51f None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.503079 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf17e8-7dc9-4000-a744-92c98c97f51f None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.503513 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf17e8-7dc9-4000-a744-92c98c97f51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.514465 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-48c33e92-5047-42d6-a78f-5cd9f2a576f9 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:08.516245 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-48c33e92-5047-42d6-a78f-5cd9f2a576f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:08.537008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 789/1556] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:08.537008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35ba73-5f59-4d19-a847-720f9db94f6d None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.537008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35ba73-5f59-4d19-a847-720f9db94f6d None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.537008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35ba73-5f59-4d19-a847-720f9db94f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.538479 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-89cf17e8-7dc9-4000-a744-92c98c97f51f None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:08.542106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-89cf17e8-7dc9-4000-a744-92c98c97f51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:08.551878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 768/1557] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:08.553012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881a03c-5589-4fa2-9a0e-0812def4e694 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.553365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881a03c-5589-4fa2-9a0e-0812def4e694 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.553719 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881a03c-5589-4fa2-9a0e-0812def4e694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.566021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-8f35ba73-5f59-4d19-a847-720f9db94f6d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:08.567244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8f35ba73-5f59-4d19-a847-720f9db94f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:08.573619 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-7881a03c-5589-4fa2-9a0e-0812def4e694 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:08.574898 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-7881a03c-5589-4fa2-9a0e-0812def4e694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:08.576024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 790/1558] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:08.577150 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebade38-9e9c-4d6a-a9da-1eab816f68eb None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.577432 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebade38-9e9c-4d6a-a9da-1eab816f68eb None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.577718 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebade38-9e9c-4d6a-a9da-1eab816f68eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.589576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 769/1559] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:08.591071 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881a03c-5589-4fa2-9a0e-0812def4e694 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:08.599515 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-2ebade38-9e9c-4d6a-a9da-1eab816f68eb None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:08.600722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-2ebade38-9e9c-4d6a-a9da-1eab816f68eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:08.603725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881a03c-5589-4fa2-9a0e-0812def4e694 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:08.609093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 791/1560] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:08.610272 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23b2c0-c813-442f-a892-edba5dbce92e None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.610553 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23b2c0-c813-442f-a892-edba5dbce92e None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.610821 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23b2c0-c813-442f-a892-edba5dbce92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.613034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c86b1-e4c6-49ca-b1c6-33b4ca1dca7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:08.614154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c86b1-e4c6-49ca-b1c6-33b4ca1dca7d admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.614720 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c86b1-e4c6-49ca-b1c6-33b4ca1dca7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.615280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c86b1-e4c6-49ca-b1c6-33b4ca1dca7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.616963 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c86b1-e4c6-49ca-b1c6-33b4ca1dca7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:08.620933 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c86b1-e4c6-49ca-b1c6-33b4ca1dca7d admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:08.630852 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-ae23b2c0-c813-442f-a892-edba5dbce92e None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:08.632087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ae23b2c0-c813-442f-a892-edba5dbce92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:08.632876 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 770/1561] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:08.637730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed41c00-8ee7-4937-8249-4c19b2b63939 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.638057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed41c00-8ee7-4937-8249-4c19b2b63939 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.639508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed41c00-8ee7-4937-8249-4c19b2b63939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.641627 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 792/1562] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:08.642864 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1b2b1-19c4-4524-82eb-7cdd51d1383d None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.643109 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1b2b1-19c4-4524-82eb-7cdd51d1383d None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.643380 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1b2b1-19c4-4524-82eb-7cdd51d1383d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.663862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-bed41c00-8ee7-4937-8249-4c19b2b63939 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:08.665098 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-bed41c00-8ee7-4937-8249-4c19b2b63939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:08.668646 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-bdf1b2b1-19c4-4524-82eb-7cdd51d1383d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:08.670085 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-bdf1b2b1-19c4-4524-82eb-7cdd51d1383d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:08.675530 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 771/1563] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:08.675530 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a25ab-886e-4be6-8139-7e1fcc4a2f4d None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.675530 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a25ab-886e-4be6-8139-7e1fcc4a2f4d None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.675893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a25ab-886e-4be6-8139-7e1fcc4a2f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.685655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 793/1564] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:08.688563 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1b2b1-19c4-4524-82eb-7cdd51d1383d None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:08.706518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-d73a25ab-886e-4be6-8139-7e1fcc4a2f4d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:08.707906 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d73a25ab-886e-4be6-8139-7e1fcc4a2f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:08.714148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1b2b1-19c4-4524-82eb-7cdd51d1383d None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:08.719466 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 772/1565] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:08.719868 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12de684-6d7e-4ff5-96c7-7f865b8363b6 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.720153 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12de684-6d7e-4ff5-96c7-7f865b8363b6 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.720454 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12de684-6d7e-4ff5-96c7-7f865b8363b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.723927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4a4e6-88b0-42c0-b044-ac3e7a186b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:08.724685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4a4e6-88b0-42c0-b044-ac3e7a186b3b admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.725063 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4a4e6-88b0-42c0-b044-ac3e7a186b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.725456 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4a4e6-88b0-42c0-b044-ac3e7a186b3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.726204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4a4e6-88b0-42c0-b044-ac3e7a186b3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:12:08.726584 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4a4e6-88b0-42c0-b044-ac3e7a186b3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:08.729259 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4a4e6-88b0-42c0-b044-ac3e7a186b3b admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:08.741879 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-c12de684-6d7e-4ff5-96c7-7f865b8363b6 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:08.743276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c12de684-6d7e-4ff5-96c7-7f865b8363b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:08.744453 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 794/1566] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:12:08.745433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4a4e6-88b0-42c0-b044-ac3e7a186b3b admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:08.760311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 773/1567] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:08.762166 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12de684-6d7e-4ff5-96c7-7f865b8363b6 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:08.767779 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4a4e6-88b0-42c0-b044-ac3e7a186b3b admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:08.773120 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f223150-6f47-4ce1-ba59-1be7970825ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:08.773734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f223150-6f47-4ce1-ba59-1be7970825ab admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.774013 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f223150-6f47-4ce1-ba59-1be7970825ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.774283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f223150-6f47-4ce1-ba59-1be7970825ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.774940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f223150-6f47-4ce1-ba59-1be7970825ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:08.775642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12de684-6d7e-4ff5-96c7-7f865b8363b6 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:08.777248 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f223150-6f47-4ce1-ba59-1be7970825ab admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:08.781434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48134a-f307-400b-a511-939db9b9cc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:08.782820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48134a-f307-400b-a511-939db9b9cc5e admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.783320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48134a-f307-400b-a511-939db9b9cc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.783621 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48134a-f307-400b-a511-939db9b9cc5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.784859 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48134a-f307-400b-a511-939db9b9cc5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:12:08.785398 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48134a-f307-400b-a511-939db9b9cc5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:08.786675 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: WARNING py.warnings [None req-ec48134a-f307-400b-a511-939db9b9cc5e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 07:12:08.786675 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: warnings.warn(msg) Aug 23 07:12:08.786675 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: Aug 23 07:12:08.788277 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48134a-f307-400b-a511-939db9b9cc5e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:08.800578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 795/1568] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:08.800578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c7172-9824-4c69-941a-464fcb1b1d13 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.801137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c7172-9824-4c69-941a-464fcb1b1d13 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.801628 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c7172-9824-4c69-941a-464fcb1b1d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.805731 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 774/1569] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:12:08.807015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48134a-f307-400b-a511-939db9b9cc5e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:08.825017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48134a-f307-400b-a511-939db9b9cc5e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:08.828807 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-4b3c7172-9824-4c69-941a-464fcb1b1d13 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:08.831154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4b3c7172-9824-4c69-941a-464fcb1b1d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:08.833453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11358414-9c84-4301-87bb-36c91b714991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:08.834333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11358414-9c84-4301-87bb-36c91b714991 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.834859 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11358414-9c84-4301-87bb-36c91b714991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.835325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11358414-9c84-4301-87bb-36c91b714991 admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.836096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11358414-9c84-4301-87bb-36c91b714991 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:12:08.836406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11358414-9c84-4301-87bb-36c91b714991 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:08.838879 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11358414-9c84-4301-87bb-36c91b714991 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:08.843309 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 796/1570] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:08.844272 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c7172-9824-4c69-941a-464fcb1b1d13 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:08.857025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c7172-9824-4c69-941a-464fcb1b1d13 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:08.858677 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 775/1571] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:12:08.859746 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11358414-9c84-4301-87bb-36c91b714991 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:08.862205 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ad50c-0c67-44a6-921e-22a69506a903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:08.862915 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ad50c-0c67-44a6-921e-22a69506a903 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.863264 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ad50c-0c67-44a6-921e-22a69506a903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.863638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ad50c-0c67-44a6-921e-22a69506a903 admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.864502 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ad50c-0c67-44a6-921e-22a69506a903 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:12:08.864875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ad50c-0c67-44a6-921e-22a69506a903 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:08.866563 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ad50c-0c67-44a6-921e-22a69506a903 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:08.876849 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11358414-9c84-4301-87bb-36c91b714991 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:08.880962 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 797/1572] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:12:08.883367 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89b00a-0a70-43f5-88a9-06112f5d7447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:08.884237 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89b00a-0a70-43f5-88a9-06112f5d7447 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.885947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89b00a-0a70-43f5-88a9-06112f5d7447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.886438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89b00a-0a70-43f5-88a9-06112f5d7447 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.887296 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89b00a-0a70-43f5-88a9-06112f5d7447 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:08.889442 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ad50c-0c67-44a6-921e-22a69506a903 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:08.890278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89b00a-0a70-43f5-88a9-06112f5d7447 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:08.908897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ad50c-0c67-44a6-921e-22a69506a903 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:08.913719 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bb82e-212b-44dd-b375-cc24359f221f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:08.914458 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bb82e-212b-44dd-b375-cc24359f221f admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:08.914744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bb82e-212b-44dd-b375-cc24359f221f admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:08.915087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bb82e-212b-44dd-b375-cc24359f221f admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:08.915985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bb82e-212b-44dd-b375-cc24359f221f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:12:08.916392 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bb82e-212b-44dd-b375-cc24359f221f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:08.918692 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bb82e-212b-44dd-b375-cc24359f221f admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:08.927181 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 776/1573] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:08 2022] POST /identity/v3/projects => generated 357 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:08.928514 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89b00a-0a70-43f5-88a9-06112f5d7447 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:08.934258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 798/1574] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:12:08.942566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bb82e-212b-44dd-b375-cc24359f221f admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:08.954618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bb82e-212b-44dd-b375-cc24359f221f admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:09.020488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89b00a-0a70-43f5-88a9-06112f5d7447 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:09.027069 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d4555-2091-4ad4-aa90-d36644b47715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:09.028159 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d4555-2091-4ad4-aa90-d36644b47715 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:09.028846 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d4555-2091-4ad4-aa90-d36644b47715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:09.029316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d4555-2091-4ad4-aa90-d36644b47715 admin admin] PATH_INFO: `/v3/projects/24307daab4d24f979c30e081163c3bfe/users/6cbcc9668f454c0e8c27495a1827d703/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:09.030270 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389e665-dbe4-436e-ac76-9839d982701f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:09.030924 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389e665-dbe4-436e-ac76-9839d982701f admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:09.031179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389e665-dbe4-436e-ac76-9839d982701f admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:09.031476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389e665-dbe4-436e-ac76-9839d982701f admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:09.032221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389e665-dbe4-436e-ac76-9839d982701f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:09.035334 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389e665-dbe4-436e-ac76-9839d982701f admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:09.049874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d4555-2091-4ad4-aa90-d36644b47715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24307daab4d24f979c30e081163c3bfe, user_id=6cbcc9668f454c0e8c27495a1827d703, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:09.053368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d4555-2091-4ad4-aa90-d36644b47715 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:09.068304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 777/1575] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:08 2022] PUT /identity/v3/projects/24307daab4d24f979c30e081163c3bfe/users/6cbcc9668f454c0e8c27495a1827d703/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:09.069982 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d4555-2091-4ad4-aa90-d36644b47715 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:09.077118 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 799/1576] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:08 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:09.077848 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389e665-dbe4-436e-ac76-9839d982701f admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:09.091291 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:09.110592 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:09.154064 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d4555-2091-4ad4-aa90-d36644b47715 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:09.166335 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389e665-dbe4-436e-ac76-9839d982701f admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:09.172780 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc845b-ceec-417d-aeaa-2979ef1735bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:09.173735 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc845b-ceec-417d-aeaa-2979ef1735bb admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:09.174211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc845b-ceec-417d-aeaa-2979ef1735bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:09.174735 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc845b-ceec-417d-aeaa-2979ef1735bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:09.175388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc845b-ceec-417d-aeaa-2979ef1735bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:09.178143 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc845b-ceec-417d-aeaa-2979ef1735bb admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:09.216828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 800/1577] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:09 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:09.218530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc845b-ceec-417d-aeaa-2979ef1735bb admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:09.239246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1d6e7-f455-48fd-868b-8a3747eaf73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:09.240194 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1d6e7-f455-48fd-868b-8a3747eaf73a admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:09.240639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1d6e7-f455-48fd-868b-8a3747eaf73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:09.240910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1d6e7-f455-48fd-868b-8a3747eaf73a admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:09.241950 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1d6e7-f455-48fd-868b-8a3747eaf73a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:12:09.242413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1d6e7-f455-48fd-868b-8a3747eaf73a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:09.244990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1d6e7-f455-48fd-868b-8a3747eaf73a admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:09.268216 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 778/1578] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:12:09.270475 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1d6e7-f455-48fd-868b-8a3747eaf73a admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:09.289723 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1d6e7-f455-48fd-868b-8a3747eaf73a admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:09.324466 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc845b-ceec-417d-aeaa-2979ef1735bb admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:09.331353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28d446-b39c-4889-b63e-a8912a0c7f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:09.332381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28d446-b39c-4889-b63e-a8912a0c7f35 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:09.332559 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28d446-b39c-4889-b63e-a8912a0c7f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:09.332833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28d446-b39c-4889-b63e-a8912a0c7f35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:09.333528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28d446-b39c-4889-b63e-a8912a0c7f35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:09.335633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28d446-b39c-4889-b63e-a8912a0c7f35 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:09.383537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 801/1579] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:09 2022] POST /identity/v3/projects => generated 375 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:09.384664 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3d57f-487b-4c16-8f82-67dd16cdc05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:09.384918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28d446-b39c-4889-b63e-a8912a0c7f35 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:09.385474 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3d57f-487b-4c16-8f82-67dd16cdc05c admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:09.385763 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3d57f-487b-4c16-8f82-67dd16cdc05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:09.386039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3d57f-487b-4c16-8f82-67dd16cdc05c admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:09.386675 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3d57f-487b-4c16-8f82-67dd16cdc05c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:09.388046 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: WARNING py.warnings [None req-e2e3d57f-487b-4c16-8f82-67dd16cdc05c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 07:12:09.388046 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: warnings.warn(msg) Aug 23 07:12:09.388046 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: Aug 23 07:12:09.389322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3d57f-487b-4c16-8f82-67dd16cdc05c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:09.422267 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 779/1580] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:09 2022] POST /identity/v3/users => generated 279 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:09.423841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fae4b-9685-4485-b40f-74f6c7d56cfa None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:09.424131 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fae4b-9685-4485-b40f-74f6c7d56cfa None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:09.424410 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fae4b-9685-4485-b40f-74f6c7d56cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:09.450045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-164fae4b-9685-4485-b40f-74f6c7d56cfa None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:09.451328 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-164fae4b-9685-4485-b40f-74f6c7d56cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:09.487043 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28d446-b39c-4889-b63e-a8912a0c7f35 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:09.493109 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b385a0-3c34-47a1-841e-aca7a0659430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:09.493904 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b385a0-3c34-47a1-841e-aca7a0659430 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:09.494187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b385a0-3c34-47a1-841e-aca7a0659430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:09.494947 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b385a0-3c34-47a1-841e-aca7a0659430 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:09.495761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b385a0-3c34-47a1-841e-aca7a0659430 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:09.498109 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b385a0-3c34-47a1-841e-aca7a0659430 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:09.532706 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:09.537363 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 802/1581] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:09 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:09.538822 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b385a0-3c34-47a1-841e-aca7a0659430 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:09.544894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:09.552994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 780/1582] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:09.554231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fae4b-9685-4485-b40f-74f6c7d56cfa None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:09.566355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fae4b-9685-4485-b40f-74f6c7d56cfa None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:09.635875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b385a0-3c34-47a1-841e-aca7a0659430 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:09.642382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2ac8d-01f3-4fa1-94c8-0967f209d8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:09.643148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2ac8d-01f3-4fa1-94c8-0967f209d8dc admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:09.644413 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2ac8d-01f3-4fa1-94c8-0967f209d8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:09.644413 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2ac8d-01f3-4fa1-94c8-0967f209d8dc admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:09.644413 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b2ac8d-01f3-4fa1-94c8-0967f209d8dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:09.646455 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b2ac8d-01f3-4fa1-94c8-0967f209d8dc admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:09.658591 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203ae0c-c331-4ac2-8085-8c384a02066e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:09.659551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203ae0c-c331-4ac2-8085-8c384a02066e admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:09.659842 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203ae0c-c331-4ac2-8085-8c384a02066e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:09.660120 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203ae0c-c331-4ac2-8085-8c384a02066e admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:09.660884 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0203ae0c-c331-4ac2-8085-8c384a02066e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:09.663082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0203ae0c-c331-4ac2-8085-8c384a02066e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:09.690689 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 803/1583] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:09 2022] POST /identity/v3/users => generated 285 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:09.692023 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2ac8d-01f3-4fa1-94c8-0967f209d8dc admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:09.698980 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 781/1584] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:09 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:09.698980 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203ae0c-c331-4ac2-8085-8c384a02066e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:09.707486 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2ac8d-01f3-4fa1-94c8-0967f209d8dc admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:09.711658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203ae0c-c331-4ac2-8085-8c384a02066e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:09.717379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682ddc1-d7a9-4236-a205-41f20d74b488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:09.718350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682ddc1-d7a9-4236-a205-41f20d74b488 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:09.718350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682ddc1-d7a9-4236-a205-41f20d74b488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:09.718635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682ddc1-d7a9-4236-a205-41f20d74b488 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:09.719161 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4682ddc1-d7a9-4236-a205-41f20d74b488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:09.721147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4682ddc1-d7a9-4236-a205-41f20d74b488 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:09.723051 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2695905-c381-45b7-ac6f-bbcd1b8f2938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:09.724243 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2695905-c381-45b7-ac6f-bbcd1b8f2938 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:09.724618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2695905-c381-45b7-ac6f-bbcd1b8f2938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:09.725074 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2695905-c381-45b7-ac6f-bbcd1b8f2938 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:09.726292 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2695905-c381-45b7-ac6f-bbcd1b8f2938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:09.728439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2695905-c381-45b7-ac6f-bbcd1b8f2938 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:09.757914 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 782/1585] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:09 2022] POST /identity/v3/users => generated 288 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:09.759461 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2695905-c381-45b7-ac6f-bbcd1b8f2938 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:09.764622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 804/1586] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:09 2022] POST /identity/v3/projects => generated 371 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:09.766291 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682ddc1-d7a9-4236-a205-41f20d74b488 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:09.772465 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2695905-c381-45b7-ac6f-bbcd1b8f2938 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:09.864871 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684081b-438e-4aa9-880d-14112176952e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:09.865505 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684081b-438e-4aa9-880d-14112176952e admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:09.865791 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684081b-438e-4aa9-880d-14112176952e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:09.866168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684081b-438e-4aa9-880d-14112176952e admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:09.866716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684081b-438e-4aa9-880d-14112176952e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:09.868442 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684081b-438e-4aa9-880d-14112176952e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:09.870226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682ddc1-d7a9-4236-a205-41f20d74b488 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:09.875875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6424e-9113-42ee-b0ae-03a351504ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:09.876243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6424e-9113-42ee-b0ae-03a351504ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:09.876522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6424e-9113-42ee-b0ae-03a351504ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:09.876791 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6424e-9113-42ee-b0ae-03a351504ca5 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:09.877421 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6424e-9113-42ee-b0ae-03a351504ca5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:09.879484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6424e-9113-42ee-b0ae-03a351504ca5 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:09.880137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 783/1587] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:09.881080 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684081b-438e-4aa9-880d-14112176952e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:09.916571 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 805/1588] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:09 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:09.917751 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6424e-9113-42ee-b0ae-03a351504ca5 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:09.934164 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6424e-9113-42ee-b0ae-03a351504ca5 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:09.943333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6059a-dd8c-4ac9-b9d9-d2ac71d439fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:09.944860 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6059a-dd8c-4ac9-b9d9-d2ac71d439fb admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:09.946122 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6059a-dd8c-4ac9-b9d9-d2ac71d439fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:09.946122 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6059a-dd8c-4ac9-b9d9-d2ac71d439fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:09.947462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6059a-dd8c-4ac9-b9d9-d2ac71d439fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:09.950674 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6059a-dd8c-4ac9-b9d9-d2ac71d439fb admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:09.967048 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 806/1589] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:09.968475 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6059a-dd8c-4ac9-b9d9-d2ac71d439fb admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:09.985707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684081b-438e-4aa9-880d-14112176952e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:09.989838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6059a-dd8c-4ac9-b9d9-d2ac71d439fb admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:09.995778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3f04e-d264-42b1-8f2c-92b8ea7b5a8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:09.996535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3f04e-d264-42b1-8f2c-92b8ea7b5a8e service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:09.996884 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3f04e-d264-42b1-8f2c-92b8ea7b5a8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:09.997073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5339c-c155-4622-8b87-b24a26c9b0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:09.997290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3f04e-d264-42b1-8f2c-92b8ea7b5a8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:09.998076 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5339c-c155-4622-8b87-b24a26c9b0ed admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:09.998582 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5339c-c155-4622-8b87-b24a26c9b0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:09.998998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5339c-c155-4622-8b87-b24a26c9b0ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:09.999942 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5339c-c155-4622-8b87-b24a26c9b0ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:10.001815 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5339c-c155-4622-8b87-b24a26c9b0ed admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:10.013421 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3f04e-d264-42b1-8f2c-92b8ea7b5a8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:10.015559 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3f04e-d264-42b1-8f2c-92b8ea7b5a8e service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:10.019301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 807/1590] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:10.019301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5339c-c155-4622-8b87-b24a26c9b0ed admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:10.028383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 784/1591] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:10.029924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3f04e-d264-42b1-8f2c-92b8ea7b5a8e service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:10.042706 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5339c-c155-4622-8b87-b24a26c9b0ed admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:10.043167 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3f04e-d264-42b1-8f2c-92b8ea7b5a8e service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:10.050688 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d3536-3fe8-479e-9cbf-e95d5fad88c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:10.051455 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d3536-3fe8-479e-9cbf-e95d5fad88c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:10.051832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d3536-3fe8-479e-9cbf-e95d5fad88c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:10.052428 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bba29-b52e-4d07-a2d6-16c476950e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:10.052545 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d3536-3fe8-479e-9cbf-e95d5fad88c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:10.053259 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d3536-3fe8-479e-9cbf-e95d5fad88c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:10.054083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bba29-b52e-4d07-a2d6-16c476950e0d admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:10.054759 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bba29-b52e-4d07-a2d6-16c476950e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:10.055216 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d3536-3fe8-479e-9cbf-e95d5fad88c0 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:10.056521 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bba29-b52e-4d07-a2d6-16c476950e0d admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:10.058821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bba29-b52e-4d07-a2d6-16c476950e0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:10.062783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bba29-b52e-4d07-a2d6-16c476950e0d admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:10.069153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 808/1592] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:10.070229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d3536-3fe8-479e-9cbf-e95d5fad88c0 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:10.080146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:10.088159 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d3536-3fe8-479e-9cbf-e95d5fad88c0 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:10.097388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c4001-9dbb-4f97-83c3-61a946e9245f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:10.098286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c4001-9dbb-4f97-83c3-61a946e9245f admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:10.098715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c4001-9dbb-4f97-83c3-61a946e9245f admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:10.102275 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c4001-9dbb-4f97-83c3-61a946e9245f admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:10.102423 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 785/1593] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:10 2022] POST /identity/v3/users => generated 286 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:10.102511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c4001-9dbb-4f97-83c3-61a946e9245f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:10.102601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bba29-b52e-4d07-a2d6-16c476950e0d admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:10.102692 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c4001-9dbb-4f97-83c3-61a946e9245f admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:10.115021 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bba29-b52e-4d07-a2d6-16c476950e0d admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:10.115319 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 809/1594] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:10.116302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c4001-9dbb-4f97-83c3-61a946e9245f admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:10.126022 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a738eeb-4a2b-4a16-9e78-861a20e97f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:10.126747 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a738eeb-4a2b-4a16-9e78-861a20e97f60 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:10.127048 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a738eeb-4a2b-4a16-9e78-861a20e97f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:10.127326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a738eeb-4a2b-4a16-9e78-861a20e97f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:10.128103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a738eeb-4a2b-4a16-9e78-861a20e97f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:10.129113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c4001-9dbb-4f97-83c3-61a946e9245f admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:10.130178 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a738eeb-4a2b-4a16-9e78-861a20e97f60 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:10.136537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d2407-58ec-4d64-9b6d-c00c3a8622e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:10.137743 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d2407-58ec-4d64-9b6d-c00c3a8622e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:10.138024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d2407-58ec-4d64-9b6d-c00c3a8622e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:10.138299 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d2407-58ec-4d64-9b6d-c00c3a8622e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:10.139376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d2407-58ec-4d64-9b6d-c00c3a8622e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:10.144205 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 786/1595] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:10.145395 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a738eeb-4a2b-4a16-9e78-861a20e97f60 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:10.147567 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d2407-58ec-4d64-9b6d-c00c3a8622e4 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:10.165242 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a738eeb-4a2b-4a16-9e78-861a20e97f60 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:10.172116 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a9d5e-f93b-4852-b4ef-6c34600a7326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:10.173043 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a9d5e-f93b-4852-b4ef-6c34600a7326 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:10.173501 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a9d5e-f93b-4852-b4ef-6c34600a7326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:10.173869 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a9d5e-f93b-4852-b4ef-6c34600a7326 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:10.174661 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a9d5e-f93b-4852-b4ef-6c34600a7326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:10.176732 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a9d5e-f93b-4852-b4ef-6c34600a7326 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:10.182421 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 810/1596] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:10.183128 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d2407-58ec-4d64-9b6d-c00c3a8622e4 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:10.202272 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:10.209144 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d2407-58ec-4d64-9b6d-c00c3a8622e4 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:10.212072 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 787/1597] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:10.213752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a9d5e-f93b-4852-b4ef-6c34600a7326 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:10.224651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a3faa-92ad-41e4-9bce-a277318f7353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:10.225339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a3faa-92ad-41e4-9bce-a277318f7353 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:10.225669 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a3faa-92ad-41e4-9bce-a277318f7353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:10.225945 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a3faa-92ad-41e4-9bce-a277318f7353 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:10.226678 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a3faa-92ad-41e4-9bce-a277318f7353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:10.228445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a3faa-92ad-41e4-9bce-a277318f7353 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:10.245510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a9d5e-f93b-4852-b4ef-6c34600a7326 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:10.252244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 811/1598] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:10.256283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942537f-6f66-45e1-8e67-04f6981a3fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:10.257809 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a3faa-92ad-41e4-9bce-a277318f7353 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:10.257996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942537f-6f66-45e1-8e67-04f6981a3fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:10.258144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942537f-6f66-45e1-8e67-04f6981a3fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:10.258585 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942537f-6f66-45e1-8e67-04f6981a3fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:10.259331 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942537f-6f66-45e1-8e67-04f6981a3fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:10.262443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942537f-6f66-45e1-8e67-04f6981a3fb2 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:10.273611 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a3faa-92ad-41e4-9bce-a277318f7353 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:10.279791 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 788/1599] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:10.279791 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942537f-6f66-45e1-8e67-04f6981a3fb2 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:10.280200 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494c14e-e9dc-4cac-b0a2-04a35bebabbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:10.280200 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494c14e-e9dc-4cac-b0a2-04a35bebabbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:10.280402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494c14e-e9dc-4cac-b0a2-04a35bebabbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:10.280501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494c14e-e9dc-4cac-b0a2-04a35bebabbe admin admin] PATH_INFO: `/v3/projects/72debe469f5744d698f5967399563c7a/users/78b086e77cca4868ba85210397ff94e4/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:10.308814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494c14e-e9dc-4cac-b0a2-04a35bebabbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=72debe469f5744d698f5967399563c7a, user_id=78b086e77cca4868ba85210397ff94e4, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:10.309880 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: WARNING py.warnings [None req-a494c14e-e9dc-4cac-b0a2-04a35bebabbe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 07:12:10.309880 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: warnings.warn(msg) Aug 23 07:12:10.309880 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: Aug 23 07:12:10.311922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494c14e-e9dc-4cac-b0a2-04a35bebabbe admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:10.328529 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942537f-6f66-45e1-8e67-04f6981a3fb2 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:10.344375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c33d0e-21c5-498e-8ce4-b4d1b0ac68a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:10.345223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c33d0e-21c5-498e-8ce4-b4d1b0ac68a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:10.345606 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c33d0e-21c5-498e-8ce4-b4d1b0ac68a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:10.345951 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c33d0e-21c5-498e-8ce4-b4d1b0ac68a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:10.346998 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c33d0e-21c5-498e-8ce4-b4d1b0ac68a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:10.349923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c33d0e-21c5-498e-8ce4-b4d1b0ac68a0 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:10.354670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 812/1600] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:10 2022] PUT /identity/v3/projects/72debe469f5744d698f5967399563c7a/users/78b086e77cca4868ba85210397ff94e4/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:10.355294 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494c14e-e9dc-4cac-b0a2-04a35bebabbe admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:10.373564 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 789/1601] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:10.373564 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c33d0e-21c5-498e-8ce4-b4d1b0ac68a0 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:10.388142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c33d0e-21c5-498e-8ce4-b4d1b0ac68a0 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:10.493338 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852097ec-0bc9-4c68-9179-351ab6302886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:10.494484 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852097ec-0bc9-4c68-9179-351ab6302886 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:10.494746 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852097ec-0bc9-4c68-9179-351ab6302886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:10.495457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852097ec-0bc9-4c68-9179-351ab6302886 admin admin] PATH_INFO: `/v3/projects/2877fe45fc8a4710a294486a9c1991b8/users/4cf8918a05f8452a9bb8e17a47de7a8a/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:10.496389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494c14e-e9dc-4cac-b0a2-04a35bebabbe admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:10.508814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bda2ab-982a-4056-b0b8-3f2a52919450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:10.510471 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bda2ab-982a-4056-b0b8-3f2a52919450 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:10.510776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bda2ab-982a-4056-b0b8-3f2a52919450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:10.511557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bda2ab-982a-4056-b0b8-3f2a52919450 admin admin] PATH_INFO: `/v3/projects/f1dc52471ac841d5be9887ddfd1503c1/users/326bc6f8519c4497b1a7d7caeb6b4a77/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:10.526592 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852097ec-0bc9-4c68-9179-351ab6302886 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2877fe45fc8a4710a294486a9c1991b8, user_id=4cf8918a05f8452a9bb8e17a47de7a8a, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:10.638369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bda2ab-982a-4056-b0b8-3f2a52919450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1dc52471ac841d5be9887ddfd1503c1, user_id=326bc6f8519c4497b1a7d7caeb6b4a77, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:10.639580 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852097ec-0bc9-4c68-9179-351ab6302886 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:10.640331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bda2ab-982a-4056-b0b8-3f2a52919450 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:10.640331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 813/1602] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:10 2022] PUT /identity/v3/projects/f1dc52471ac841d5be9887ddfd1503c1/users/326bc6f8519c4497b1a7d7caeb6b4a77/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:10.640507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 790/1603] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:10 2022] PUT /identity/v3/projects/2877fe45fc8a4710a294486a9c1991b8/users/4cf8918a05f8452a9bb8e17a47de7a8a/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:10.640507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852097ec-0bc9-4c68-9179-351ab6302886 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:10.640694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bda2ab-982a-4056-b0b8-3f2a52919450 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:10.720240 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852097ec-0bc9-4c68-9179-351ab6302886 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:10.750596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bda2ab-982a-4056-b0b8-3f2a52919450 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:10.760406 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edabd5-9a71-48bb-ae35-ace8ca67ea3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:10.761373 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edabd5-9a71-48bb-ae35-ace8ca67ea3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:10.761749 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edabd5-9a71-48bb-ae35-ace8ca67ea3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:10.762223 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edabd5-9a71-48bb-ae35-ace8ca67ea3b admin admin] PATH_INFO: `/v3/projects/e449a86b7ec3494583203cb6abc4a930/users/c31f51896f9a492298ee8185aba9a480/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:10.779766 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:10.788816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edabd5-9a71-48bb-ae35-ace8ca67ea3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e449a86b7ec3494583203cb6abc4a930, user_id=c31f51896f9a492298ee8185aba9a480, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:10.791892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edabd5-9a71-48bb-ae35-ace8ca67ea3b admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:10.813576 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 814/1604] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:10 2022] PUT /identity/v3/projects/e449a86b7ec3494583203cb6abc4a930/users/c31f51896f9a492298ee8185aba9a480/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:10.827880 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6228f-46de-438c-8c7b-b28d2aadd9fd None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:10.828168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6228f-46de-438c-8c7b-b28d2aadd9fd None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:10.828521 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6228f-46de-438c-8c7b-b28d2aadd9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:10.836112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f8366-c5d8-4355-9502-fde3bc67c3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:10.836805 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f8366-c5d8-4355-9502-fde3bc67c3fd admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:10.837084 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f8366-c5d8-4355-9502-fde3bc67c3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:10.837403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f8366-c5d8-4355-9502-fde3bc67c3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:10.838481 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f8366-c5d8-4355-9502-fde3bc67c3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:10.840231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f8366-c5d8-4355-9502-fde3bc67c3fd admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:10.857229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 791/1605] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:10.857229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f8366-c5d8-4355-9502-fde3bc67c3fd admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:10.869038 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-2ec6228f-46de-438c-8c7b-b28d2aadd9fd None None] MFA Rules not processed for user `78b086e77cca4868ba85210397ff94e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:10.869323 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f8366-c5d8-4355-9502-fde3bc67c3fd admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:10.870971 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-2ec6228f-46de-438c-8c7b-b28d2aadd9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:10.903180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:10.967978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc14f1-0ba1-447b-baf7-fb11776fa761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:10.969916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc14f1-0ba1-447b-baf7-fb11776fa761 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:10.970413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc14f1-0ba1-447b-baf7-fb11776fa761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:10.970824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc14f1-0ba1-447b-baf7-fb11776fa761 admin admin] PATH_INFO: `/v3/projects/eb44f62fa62e46528712f24479cbb19f/users/471dc5c1db64411ab1d80ad6c038e519/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:10.980513 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 815/1606] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:10.982457 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06207a5e-2e86-4cbf-9a92-7e85169c660f None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:10.982892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06207a5e-2e86-4cbf-9a92-7e85169c660f None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:10.983206 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06207a5e-2e86-4cbf-9a92-7e85169c660f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:10.995678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc14f1-0ba1-447b-baf7-fb11776fa761 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb44f62fa62e46528712f24479cbb19f, user_id=471dc5c1db64411ab1d80ad6c038e519, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:11.000075 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc14f1-0ba1-447b-baf7-fb11776fa761 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:11.029819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 792/1607] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:10 2022] PUT /identity/v3/projects/eb44f62fa62e46528712f24479cbb19f/users/471dc5c1db64411ab1d80ad6c038e519/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:11.032227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd7f3d-c42d-4ee3-abb9-88b2b676f7d8 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:11.032509 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd7f3d-c42d-4ee3-abb9-88b2b676f7d8 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:11.033617 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd7f3d-c42d-4ee3-abb9-88b2b676f7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:11.036405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-06207a5e-2e86-4cbf-9a92-7e85169c660f None None] MFA Rules not processed for user `326bc6f8519c4497b1a7d7caeb6b4a77`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:11.038896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-06207a5e-2e86-4cbf-9a92-7e85169c660f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:11.085211 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-98cd7f3d-c42d-4ee3-abb9-88b2b676f7d8 None None] MFA Rules not processed for user `4cf8918a05f8452a9bb8e17a47de7a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:11.086429 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-98cd7f3d-c42d-4ee3-abb9-88b2b676f7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:11.178816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 793/1608] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:11.178816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945aadc-84da-4c41-9560-bfbebc06d84e None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:11.178816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945aadc-84da-4c41-9560-bfbebc06d84e None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:11.178816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945aadc-84da-4c41-9560-bfbebc06d84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:11.189908 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 816/1609] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:11.191829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06207a5e-2e86-4cbf-9a92-7e85169c660f None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:11.231935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-c945aadc-84da-4c41-9560-bfbebc06d84e None None] MFA Rules not processed for user `c31f51896f9a492298ee8185aba9a480`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:11.234599 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c945aadc-84da-4c41-9560-bfbebc06d84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:11.309790 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:11.325489 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06207a5e-2e86-4cbf-9a92-7e85169c660f None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:11.336858 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b3045-a88f-4387-a80a-0aa456b23d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:11.337635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b3045-a88f-4387-a80a-0aa456b23d78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:11.337896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b3045-a88f-4387-a80a-0aa456b23d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:11.338598 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b3045-a88f-4387-a80a-0aa456b23d78 admin admin] PATH_INFO: `/v3/projects/3b1d47921fb14648b8da10af27d12ce4/users/7d1401598f1b4cf7b7c3e89dfbd0debf/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:11.377051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b3045-a88f-4387-a80a-0aa456b23d78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b1d47921fb14648b8da10af27d12ce4, user_id=7d1401598f1b4cf7b7c3e89dfbd0debf, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:11.379783 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b3045-a88f-4387-a80a-0aa456b23d78 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:11.395045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 794/1610] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:11.402747 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945aadc-84da-4c41-9560-bfbebc06d84e None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:11.413315 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 817/1611] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:11 2022] PUT /identity/v3/projects/3b1d47921fb14648b8da10af27d12ce4/users/7d1401598f1b4cf7b7c3e89dfbd0debf/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:11.416048 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c8738-259b-4746-9d8c-3542d0da3e2f None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:11.416552 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c8738-259b-4746-9d8c-3542d0da3e2f None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:11.416941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c8738-259b-4746-9d8c-3542d0da3e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:11.443477 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945aadc-84da-4c41-9560-bfbebc06d84e None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:11.472032 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-362c8738-259b-4746-9d8c-3542d0da3e2f None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:11.475286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-362c8738-259b-4746-9d8c-3542d0da3e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:11.544603 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:11.571894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbaed1-fec5-44d6-bcb7-3cec95352e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:11.574062 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbaed1-fec5-44d6-bcb7-3cec95352e9f admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:11.576063 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbaed1-fec5-44d6-bcb7-3cec95352e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:11.576542 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbaed1-fec5-44d6-bcb7-3cec95352e9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:11.577391 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbaed1-fec5-44d6-bcb7-3cec95352e9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:11.584031 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbaed1-fec5-44d6-bcb7-3cec95352e9f admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:11.598205 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 818/1612] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:11.611789 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f919ce1-7dbf-4cf2-9b1c-4665af24a2e6 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:11.612129 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f919ce1-7dbf-4cf2-9b1c-4665af24a2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:11.612329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f919ce1-7dbf-4cf2-9b1c-4665af24a2e6 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:11.624965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 819/1613] 10.222.0.17 () {54 vars in 962 bytes} [Tue Aug 23 07:12:11 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:12:11.626490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ab897-c5ed-47f3-8223-4237802efb40 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:11.626814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ab897-c5ed-47f3-8223-4237802efb40 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:11.627043 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ab897-c5ed-47f3-8223-4237802efb40 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:11.629480 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 820/1614] 10.222.0.17 () {54 vars in 962 bytes} [Tue Aug 23 07:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:12:11.630106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c131a5b-7721-4518-82f4-f95883d8d2b4 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:11.630418 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c131a5b-7721-4518-82f4-f95883d8d2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:11.630754 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c131a5b-7721-4518-82f4-f95883d8d2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:11.660499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 795/1615] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:11 2022] POST /identity/v3/projects => generated 357 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:11.664493 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff579e0e-28c3-4f95-a2c9-2d7c3f6d9c4f None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:11.666180 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff579e0e-28c3-4f95-a2c9-2d7c3f6d9c4f None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:11.666496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff579e0e-28c3-4f95-a2c9-2d7c3f6d9c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:11.710025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-6c131a5b-7721-4518-82f4-f95883d8d2b4 None None] MFA Rules not processed for user `7d1401598f1b4cf7b7c3e89dfbd0debf`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:11.711468 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6c131a5b-7721-4518-82f4-f95883d8d2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:11.737443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-ff579e0e-28c3-4f95-a2c9-2d7c3f6d9c4f None None] MFA Rules not processed for user `a8d2cd93042c4583b63afe629d47fce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:11.738926 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ff579e0e-28c3-4f95-a2c9-2d7c3f6d9c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:11.824363 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 796/1616] 10.222.0.17 () {58 vars in 1059 bytes} [Tue Aug 23 07:12:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:11.825994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b6149-c5aa-470c-b553-8cdfc5046795 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:11.826346 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b6149-c5aa-470c-b553-8cdfc5046795 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:11.826897 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b6149-c5aa-470c-b553-8cdfc5046795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:11.849090 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 821/1617] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:11.852926 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-386b6149-c5aa-470c-b553-8cdfc5046795 None None] MFA Rules not processed for user `a8d2cd93042c4583b63afe629d47fce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:11.854920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-386b6149-c5aa-470c-b553-8cdfc5046795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:11.856378 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c131a5b-7721-4518-82f4-f95883d8d2b4 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:11.867002 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 797/1618] 10.222.0.17 () {58 vars in 1059 bytes} [Tue Aug 23 07:12:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:11.868148 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b6149-c5aa-470c-b553-8cdfc5046795 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:11.910269 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:11.939337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b6149-c5aa-470c-b553-8cdfc5046795 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:11.945243 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477226e4-c5ec-4b5f-8138-f6bea19b254e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:11.945978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477226e4-c5ec-4b5f-8138-f6bea19b254e service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:11.946319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477226e4-c5ec-4b5f-8138-f6bea19b254e service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:11.946780 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477226e4-c5ec-4b5f-8138-f6bea19b254e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:12.000097 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c131a5b-7721-4518-82f4-f95883d8d2b4 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:12.012599 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823f6ee-796e-47af-9231-80afd04c15c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:12.013233 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823f6ee-796e-47af-9231-80afd04c15c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:12.013572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823f6ee-796e-47af-9231-80afd04c15c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:12.013780 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823f6ee-796e-47af-9231-80afd04c15c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:12.014418 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823f6ee-796e-47af-9231-80afd04c15c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:12.017602 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823f6ee-796e-47af-9231-80afd04c15c5 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:12.033449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477226e4-c5ec-4b5f-8138-f6bea19b254e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:12.035085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477226e4-c5ec-4b5f-8138-f6bea19b254e service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:12.050736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 798/1619] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:12.056054 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477226e4-c5ec-4b5f-8138-f6bea19b254e service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:12.060688 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 822/1620] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:11 2022] POST /identity/v3/users => generated 285 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:12.062457 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823f6ee-796e-47af-9231-80afd04c15c5 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:12.084769 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823f6ee-796e-47af-9231-80afd04c15c5 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:12.086435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477226e4-c5ec-4b5f-8138-f6bea19b254e service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:12.096859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b35a8-dd60-46b6-8466-5ac95fa4dd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:12.098147 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2726b-fa7e-4fb9-8416-550ff5eb87ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:12.098716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2726b-fa7e-4fb9-8416-550ff5eb87ea service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:12.098995 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2726b-fa7e-4fb9-8416-550ff5eb87ea service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:12.099984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b35a8-dd60-46b6-8466-5ac95fa4dd3d admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:12.100262 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b35a8-dd60-46b6-8466-5ac95fa4dd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:12.100711 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b35a8-dd60-46b6-8466-5ac95fa4dd3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:12.101335 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b35a8-dd60-46b6-8466-5ac95fa4dd3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:12.103339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b35a8-dd60-46b6-8466-5ac95fa4dd3d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:12.103858 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2726b-fa7e-4fb9-8416-550ff5eb87ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:12.164494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 823/1621] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:12 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:12.168711 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b35a8-dd60-46b6-8466-5ac95fa4dd3d admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:12.232226 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2726b-fa7e-4fb9-8416-550ff5eb87ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:12.235967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2726b-fa7e-4fb9-8416-550ff5eb87ea service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:12.261886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:12.262144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 799/1622] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:12.266448 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2726b-fa7e-4fb9-8416-550ff5eb87ea service nova] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:12.287294 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2726b-fa7e-4fb9-8416-550ff5eb87ea service nova] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:12.351559 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b35a8-dd60-46b6-8466-5ac95fa4dd3d admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:12.367280 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0117b5d3-42e6-4801-bf78-1070aa1e3787 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:12.367537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117b5d3-42e6-4801-bf78-1070aa1e3787 service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:12.367809 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117b5d3-42e6-4801-bf78-1070aa1e3787 service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:12.368038 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117b5d3-42e6-4801-bf78-1070aa1e3787 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:12.446779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08c7d0-a8fb-48b4-9786-8a8a5e16d3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:12.447535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08c7d0-a8fb-48b4-9786-8a8a5e16d3a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:12.452508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08c7d0-a8fb-48b4-9786-8a8a5e16d3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:12.454866 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08c7d0-a8fb-48b4-9786-8a8a5e16d3a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:12.455347 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab08c7d0-a8fb-48b4-9786-8a8a5e16d3a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:12.457537 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab08c7d0-a8fb-48b4-9786-8a8a5e16d3a8 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:12.507877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 800/1623] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:12.508850 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08c7d0-a8fb-48b4-9786-8a8a5e16d3a8 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:12.544787 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08c7d0-a8fb-48b4-9786-8a8a5e16d3a8 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:12.555328 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0117b5d3-42e6-4801-bf78-1070aa1e3787 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:12.558210 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0117b5d3-42e6-4801-bf78-1070aa1e3787 service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:12.563115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da60201-b364-4351-affa-44629e2219fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:12.564027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da60201-b364-4351-affa-44629e2219fd admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:12.564321 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da60201-b364-4351-affa-44629e2219fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:12.564881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da60201-b364-4351-affa-44629e2219fd admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:12.565378 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da60201-b364-4351-affa-44629e2219fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:12.568019 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da60201-b364-4351-affa-44629e2219fd admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:12.595450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 824/1624] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:12.596690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0117b5d3-42e6-4801-bf78-1070aa1e3787 service nova] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:12.623586 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:12.631910 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0117b5d3-42e6-4801-bf78-1070aa1e3787 service nova] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:12.641434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 801/1625] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:12 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:12.643330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da60201-b364-4351-affa-44629e2219fd admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:12.664509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc0f4c-7c8a-46d6-bb34-5f86ddbdb9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:12.665209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc0f4c-7c8a-46d6-bb34-5f86ddbdb9b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:12.665493 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc0f4c-7c8a-46d6-bb34-5f86ddbdb9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:12.666007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc0f4c-7c8a-46d6-bb34-5f86ddbdb9b3 admin admin] PATH_INFO: `/v3/projects/cbfe4f25844d4ae593c4bcc8bc55b66e/users/61208b1bbc02419a8aa708270964b2a4/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:12.693969 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da60201-b364-4351-affa-44629e2219fd admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:12.710746 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24909966-2005-49fd-8cfc-5083b1e2c875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:12.713604 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24909966-2005-49fd-8cfc-5083b1e2c875 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:12.713889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24909966-2005-49fd-8cfc-5083b1e2c875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:12.714167 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24909966-2005-49fd-8cfc-5083b1e2c875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:12.714590 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc0f4c-7c8a-46d6-bb34-5f86ddbdb9b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbfe4f25844d4ae593c4bcc8bc55b66e, user_id=61208b1bbc02419a8aa708270964b2a4, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:12.714831 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24909966-2005-49fd-8cfc-5083b1e2c875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:12.718962 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc0f4c-7c8a-46d6-bb34-5f86ddbdb9b3 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:12.724316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24909966-2005-49fd-8cfc-5083b1e2c875 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:12.747314 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 825/1626] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:12 2022] PUT /identity/v3/projects/cbfe4f25844d4ae593c4bcc8bc55b66e/users/61208b1bbc02419a8aa708270964b2a4/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:12.751352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc0f4c-7c8a-46d6-bb34-5f86ddbdb9b3 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:12.778637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 802/1627] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:12.787749 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24909966-2005-49fd-8cfc-5083b1e2c875 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:12.835496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24909966-2005-49fd-8cfc-5083b1e2c875 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:12.910264 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc0f4c-7c8a-46d6-bb34-5f86ddbdb9b3 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:12.935330 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d486e1b-067a-4ec6-8b04-d8098c879e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:12.935933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d486e1b-067a-4ec6-8b04-d8098c879e09 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:12.936607 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d486e1b-067a-4ec6-8b04-d8098c879e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:12.936895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d486e1b-067a-4ec6-8b04-d8098c879e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:12.937580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d486e1b-067a-4ec6-8b04-d8098c879e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:12.939381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d486e1b-067a-4ec6-8b04-d8098c879e09 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:12.974541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 826/1628] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:12:12.974541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe80fe-f78f-4e43-b025-5ef85b528650 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:12.974541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe80fe-f78f-4e43-b025-5ef85b528650 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:12.974541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe80fe-f78f-4e43-b025-5ef85b528650 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:12.978571 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 827/1629] 10.222.0.17 () {54 vars in 912 bytes} [Tue Aug 23 07:12:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:12:12.979547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe80fe-f78f-4e43-b025-5ef85b528650 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:13.017179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe80fe-f78f-4e43-b025-5ef85b528650 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:13.020366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e419f-b111-4637-9c9c-5ea050fe1290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:13.026990 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db093e-df28-489b-867a-8674212bd944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:13.027513 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db093e-df28-489b-867a-8674212bd944 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:13.028254 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db093e-df28-489b-867a-8674212bd944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:13.028565 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db093e-df28-489b-867a-8674212bd944 admin admin] PATH_INFO: `/v3/projects/cbfe4f25844d4ae593c4bcc8bc55b66e/users/61208b1bbc02419a8aa708270964b2a4/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:13.032884 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e419f-b111-4637-9c9c-5ea050fe1290 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:13.033393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e419f-b111-4637-9c9c-5ea050fe1290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:13.034490 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e419f-b111-4637-9c9c-5ea050fe1290 admin admin] PATH_INFO: `/v3/projects/985077d780104111a82dca33cdb43f22/users/6bb4e0fbf23341c1a900aab2e87db9e5/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:13.036273 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db093e-df28-489b-867a-8674212bd944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbfe4f25844d4ae593c4bcc8bc55b66e, user_id=61208b1bbc02419a8aa708270964b2a4, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:13.039462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db093e-df28-489b-867a-8674212bd944 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:13.067578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 828/1630] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:12 2022] PUT /identity/v3/projects/cbfe4f25844d4ae593c4bcc8bc55b66e/users/61208b1bbc02419a8aa708270964b2a4/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:13.069450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5f131-d93f-4c48-94f4-b0c86f54d8d3 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:13.069746 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5f131-d93f-4c48-94f4-b0c86f54d8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:13.070022 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5f131-d93f-4c48-94f4-b0c86f54d8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:13.093063 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e419f-b111-4637-9c9c-5ea050fe1290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=985077d780104111a82dca33cdb43f22, user_id=6bb4e0fbf23341c1a900aab2e87db9e5, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:13.095859 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e419f-b111-4637-9c9c-5ea050fe1290 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:13.121301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 803/1631] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:12 2022] PUT /identity/v3/projects/985077d780104111a82dca33cdb43f22/users/6bb4e0fbf23341c1a900aab2e87db9e5/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:13.123037 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2cf2f2-d460-4fb8-863a-5328f683af39 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:13.123816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2cf2f2-d460-4fb8-863a-5328f683af39 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:13.126549 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2cf2f2-d460-4fb8-863a-5328f683af39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:13.138383 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-36b5f131-d93f-4c48-94f4-b0c86f54d8d3 None None] MFA Rules not processed for user `a8d2cd93042c4583b63afe629d47fce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:13.139634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-36b5f131-d93f-4c48-94f4-b0c86f54d8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:13.187364 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-3b2cf2f2-d460-4fb8-863a-5328f683af39 None None] MFA Rules not processed for user `61208b1bbc02419a8aa708270964b2a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:13.189911 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3b2cf2f2-d460-4fb8-863a-5328f683af39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:13.209453 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:13.295153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 829/1632] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 23 07:12:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:13.296918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5f131-d93f-4c48-94f4-b0c86f54d8d3 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:13.353966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 804/1633] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:13 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:13.355257 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad20b0-1e60-449a-a72f-b07ddbc1e3cb None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:13.355573 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad20b0-1e60-449a-a72f-b07ddbc1e3cb None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:13.355851 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad20b0-1e60-449a-a72f-b07ddbc1e3cb None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:13.357268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 805/1634] 10.222.0.17 () {54 vars in 912 bytes} [Tue Aug 23 07:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:12:13.357887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad20b0-1e60-449a-a72f-b07ddbc1e3cb None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:13.453478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5f131-d93f-4c48-94f4-b0c86f54d8d3 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:13.466854 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ecac4-1ad4-4044-82f5-5d2f93f20598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:13.467615 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ecac4-1ad4-4044-82f5-5d2f93f20598 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:13.468031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ecac4-1ad4-4044-82f5-5d2f93f20598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:13.468398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ecac4-1ad4-4044-82f5-5d2f93f20598 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:13.470125 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ecac4-1ad4-4044-82f5-5d2f93f20598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:13.471929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ecac4-1ad4-4044-82f5-5d2f93f20598 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:13.494957 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 830/1635] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:13.496091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ecac4-1ad4-4044-82f5-5d2f93f20598 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:13.513729 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ecac4-1ad4-4044-82f5-5d2f93f20598 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:13.521606 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7cbd8c-af55-41ec-98dc-da3cd1590849 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:13.524683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cbd8c-af55-41ec-98dc-da3cd1590849 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:13.525306 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cbd8c-af55-41ec-98dc-da3cd1590849 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:13.525881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad20b0-1e60-449a-a72f-b07ddbc1e3cb None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:13.527054 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cbd8c-af55-41ec-98dc-da3cd1590849 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:13.542632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d636e-c053-47d8-aecf-32bacc60d90b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:13.543957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d636e-c053-47d8-aecf-32bacc60d90b service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:13.546241 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d636e-c053-47d8-aecf-32bacc60d90b service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:13.549344 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d636e-c053-47d8-aecf-32bacc60d90b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:13.586169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7cbd8c-af55-41ec-98dc-da3cd1590849 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:13.593518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7cbd8c-af55-41ec-98dc-da3cd1590849 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:13.605340 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d636e-c053-47d8-aecf-32bacc60d90b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:13.607132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d636e-c053-47d8-aecf-32bacc60d90b service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:13.618663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 831/1636] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:13 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:13.621552 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac2129-9409-46b7-b415-6584db185be8 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:13.622107 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac2129-9409-46b7-b415-6584db185be8 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:13.625064 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac2129-9409-46b7-b415-6584db185be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:13.642799 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:13.661681 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 806/1637] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:13.662788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d636e-c053-47d8-aecf-32bacc60d90b service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:13.689773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-a8ac2129-9409-46b7-b415-6584db185be8 None None] MFA Rules not processed for user `a8d2cd93042c4583b63afe629d47fce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:13.693755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d636e-c053-47d8-aecf-32bacc60d90b service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:13.695745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a8ac2129-9409-46b7-b415-6584db185be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:13.706408 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd22c8b-46bf-47b1-9b4e-577615168d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:13.708090 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd22c8b-46bf-47b1-9b4e-577615168d9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:13.719853 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd22c8b-46bf-47b1-9b4e-577615168d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:13.720375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd22c8b-46bf-47b1-9b4e-577615168d9b admin admin] PATH_INFO: `/v3/projects/985077d780104111a82dca33cdb43f22/users/6bb4e0fbf23341c1a900aab2e87db9e5/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:13.725755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd22c8b-46bf-47b1-9b4e-577615168d9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=985077d780104111a82dca33cdb43f22, user_id=6bb4e0fbf23341c1a900aab2e87db9e5, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:13.734031 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd22c8b-46bf-47b1-9b4e-577615168d9b admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:13.737087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 832/1638] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 23 07:12:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:13.768898 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 807/1639] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:13 2022] PUT /identity/v3/projects/985077d780104111a82dca33cdb43f22/users/6bb4e0fbf23341c1a900aab2e87db9e5/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:13.781616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade5af3-bcf2-438c-9ded-4482e79125a7 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:13.781898 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade5af3-bcf2-438c-9ded-4482e79125a7 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:13.782518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade5af3-bcf2-438c-9ded-4482e79125a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:13.848249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd22c8b-46bf-47b1-9b4e-577615168d9b admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:13.854380 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-8ade5af3-bcf2-438c-9ded-4482e79125a7 None None] MFA Rules not processed for user `6bb4e0fbf23341c1a900aab2e87db9e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:13.857737 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8ade5af3-bcf2-438c-9ded-4482e79125a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:14.028488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd22c8b-46bf-47b1-9b4e-577615168d9b admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:14.032592 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde430fb-40d0-4a80-ba51-b70906e52f83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:14.033380 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde430fb-40d0-4a80-ba51-b70906e52f83 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:14.033721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde430fb-40d0-4a80-ba51-b70906e52f83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:14.034061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde430fb-40d0-4a80-ba51-b70906e52f83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:14.092442 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 833/1640] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:14.129946 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde430fb-40d0-4a80-ba51-b70906e52f83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:14.131157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde430fb-40d0-4a80-ba51-b70906e52f83 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:14.146760 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 808/1641] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:14.691017 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade5af3-bcf2-438c-9ded-4482e79125a7 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:14.733580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade5af3-bcf2-438c-9ded-4482e79125a7 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:14.751405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63695e4-8325-4441-a76b-516a56590271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:14.752979 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63695e4-8325-4441-a76b-516a56590271 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:14.753954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63695e4-8325-4441-a76b-516a56590271 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:14.755093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63695e4-8325-4441-a76b-516a56590271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:14.822562 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:14.968679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63695e4-8325-4441-a76b-516a56590271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:14.970416 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63695e4-8325-4441-a76b-516a56590271 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:14.997241 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 834/1642] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:14 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:15.306084 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde430fb-40d0-4a80-ba51-b70906e52f83 service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:15.324381 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde430fb-40d0-4a80-ba51-b70906e52f83 service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:15.338735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa161f5c-9d7c-48e4-93ea-1202f1d7ade5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:15.340148 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa161f5c-9d7c-48e4-93ea-1202f1d7ade5 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:15.340440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa161f5c-9d7c-48e4-93ea-1202f1d7ade5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:15.341532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa161f5c-9d7c-48e4-93ea-1202f1d7ade5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:15.378462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa161f5c-9d7c-48e4-93ea-1202f1d7ade5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:15.380003 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa161f5c-9d7c-48e4-93ea-1202f1d7ade5 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:15.404182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 809/1643] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:15.467916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63695e4-8325-4441-a76b-516a56590271 service cinder] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:15.493437 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63695e4-8325-4441-a76b-516a56590271 service cinder] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:15.536397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf301c-ed14-47a1-96ca-5183326146d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:15.537200 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf301c-ed14-47a1-96ca-5183326146d2 service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:15.537671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf301c-ed14-47a1-96ca-5183326146d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:15.538217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf301c-ed14-47a1-96ca-5183326146d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:15.630043 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa161f5c-9d7c-48e4-93ea-1202f1d7ade5 service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:15.643518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa161f5c-9d7c-48e4-93ea-1202f1d7ade5 service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:15.649387 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac717b0-483c-4d55-8d6a-c62212507a22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:15.650999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac717b0-483c-4d55-8d6a-c62212507a22 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:15.657693 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac717b0-483c-4d55-8d6a-c62212507a22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:15.658130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac717b0-483c-4d55-8d6a-c62212507a22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:15.683601 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf301c-ed14-47a1-96ca-5183326146d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:15.692808 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf301c-ed14-47a1-96ca-5183326146d2 service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:15.773550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 835/1644] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:15.781608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac717b0-483c-4d55-8d6a-c62212507a22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:15.781608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac717b0-483c-4d55-8d6a-c62212507a22 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:15.796711 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 810/1645] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:12:16.088769 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf301c-ed14-47a1-96ca-5183326146d2 service nova] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:16.199755 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf301c-ed14-47a1-96ca-5183326146d2 service nova] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:16.207546 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d7800-519d-46ea-84ae-2c7cea6609c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:16.208527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d7800-519d-46ea-84ae-2c7cea6609c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:16.209088 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d7800-519d-46ea-84ae-2c7cea6609c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:16.209567 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d7800-519d-46ea-84ae-2c7cea6609c5 admin admin] PATH_INFO: `/v3/users/c31f51896f9a492298ee8185aba9a480` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:16.211221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d7800-519d-46ea-84ae-2c7cea6609c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c31f51896f9a492298ee8185aba9a480)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:16.212663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: WARNING py.warnings [None req-171d7800-519d-46ea-84ae-2c7cea6609c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 07:12:16.212663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: warnings.warn(msg) Aug 23 07:12:16.212663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: Aug 23 07:12:16.214437 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d7800-519d-46ea-84ae-2c7cea6609c5 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:16.351249 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-171d7800-519d-46ea-84ae-2c7cea6609c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c31f51896f9a492298ee8185aba9a480'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:16.352428 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-171d7800-519d-46ea-84ae-2c7cea6609c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c31f51896f9a492298ee8185aba9a480'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:16.353155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-171d7800-519d-46ea-84ae-2c7cea6609c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c31f51896f9a492298ee8185aba9a480'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:16.372756 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-171d7800-519d-46ea-84ae-2c7cea6609c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c31f51896f9a492298ee8185aba9a480'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:16.387138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:16.407298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-171d7800-519d-46ea-84ae-2c7cea6609c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c31f51896f9a492298ee8185aba9a480'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:16.435504 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 836/1646] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:12:16 2022] DELETE /identity/v3/users/c31f51896f9a492298ee8185aba9a480 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:16.440599 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8507a6-203f-432e-8a30-d9f5c3c2c832 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:16.441153 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8507a6-203f-432e-8a30-d9f5c3c2c832 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:16.441153 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8507a6-203f-432e-8a30-d9f5c3c2c832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:16.485257 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-1d8507a6-203f-432e-8a30-d9f5c3c2c832 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:16.494527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-1d8507a6-203f-432e-8a30-d9f5c3c2c832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:16.507687 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:16.593739 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 811/1647] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:16.606882 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d7800-519d-46ea-84ae-2c7cea6609c5 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:16.608388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-171d7800-519d-46ea-84ae-2c7cea6609c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:16.722375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d7800-519d-46ea-84ae-2c7cea6609c5 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:16.728154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bb11e-4449-48e2-b488-4d8c9a2e69f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:16.729830 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bb11e-4449-48e2-b488-4d8c9a2e69f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:16.731312 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bb11e-4449-48e2-b488-4d8c9a2e69f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:16.732009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bb11e-4449-48e2-b488-4d8c9a2e69f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:16.747232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bb11e-4449-48e2-b488-4d8c9a2e69f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:16.749613 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bb11e-4449-48e2-b488-4d8c9a2e69f7 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:16.770324 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 837/1648] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:17.498690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8507a6-203f-432e-8a30-d9f5c3c2c832 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:17.500589 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-1d8507a6-203f-432e-8a30-d9f5c3c2c832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:17.536455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8507a6-203f-432e-8a30-d9f5c3c2c832 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:17.548491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686366f-32ac-4c9a-9061-7867a7809943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:17.549151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686366f-32ac-4c9a-9061-7867a7809943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:17.549430 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686366f-32ac-4c9a-9061-7867a7809943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:17.549701 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686366f-32ac-4c9a-9061-7867a7809943 admin admin] PATH_INFO: `/v3/projects/e449a86b7ec3494583203cb6abc4a930` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:17.557652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686366f-32ac-4c9a-9061-7867a7809943 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e449a86b7ec3494583203cb6abc4a930)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:17.561978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: WARNING py.warnings [None req-a686366f-32ac-4c9a-9061-7867a7809943 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 07:12:17.561978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: warnings.warn(msg) Aug 23 07:12:17.561978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: Aug 23 07:12:17.584046 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f4c89-2079-4660-8556-a0b4ebe18e01 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:17.584343 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f4c89-2079-4660-8556-a0b4ebe18e01 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:17.584631 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f4c89-2079-4660-8556-a0b4ebe18e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:17.586543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686366f-32ac-4c9a-9061-7867a7809943 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:17.692261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-cc0f4c89-2079-4660-8556-a0b4ebe18e01 None None] MFA Rules not processed for user `64d54cb95b264c18b45d90822507fae1`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:17.701384 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-cc0f4c89-2079-4660-8556-a0b4ebe18e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:17.787068 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-a686366f-32ac-4c9a-9061-7867a7809943 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e449a86b7ec3494583203cb6abc4a930', 'request_id': 'req-a686366f-32ac-4c9a-9061-7867a7809943'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:17.805897 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-a686366f-32ac-4c9a-9061-7867a7809943 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e449a86b7ec3494583203cb6abc4a930', 'request_id': 'req-a686366f-32ac-4c9a-9061-7867a7809943'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:17.823082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-a686366f-32ac-4c9a-9061-7867a7809943 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e449a86b7ec3494583203cb6abc4a930', 'request_id': 'req-a686366f-32ac-4c9a-9061-7867a7809943'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:17.857493 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-a686366f-32ac-4c9a-9061-7867a7809943 admin admin] The token cache is being invalidate because project e449a86b7ec3494583203cb6abc4a930 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:12:17.858116 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-a686366f-32ac-4c9a-9061-7867a7809943 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:17.862543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-a686366f-32ac-4c9a-9061-7867a7809943 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:17.864872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 812/1649] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:12:17 2022] DELETE /identity/v3/projects/e449a86b7ec3494583203cb6abc4a930 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:17.869421 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa279aae-c8c8-45fc-b4e5-c09c9c3e3542 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:17.870028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa279aae-c8c8-45fc-b4e5-c09c9c3e3542 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:17.870488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa279aae-c8c8-45fc-b4e5-c09c9c3e3542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:17.872108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 838/1650] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:17.887760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5c1ae-6306-4a96-9baf-7e856f6d4121 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:17.892842 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5c1ae-6306-4a96-9baf-7e856f6d4121 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:17.893370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5c1ae-6306-4a96-9baf-7e856f6d4121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:17.911747 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-fa279aae-c8c8-45fc-b4e5-c09c9c3e3542 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:17.913121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-fa279aae-c8c8-45fc-b4e5-c09c9c3e3542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:17.958967 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-05e5c1ae-6306-4a96-9baf-7e856f6d4121 None None] MFA Rules not processed for user `64d54cb95b264c18b45d90822507fae1`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:17.961568 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-05e5c1ae-6306-4a96-9baf-7e856f6d4121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:18.132338 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 839/1651] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:18.141140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 813/1652] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:18.148978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b19869-26fe-4170-bf96-4562e3083069 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:18.149267 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b19869-26fe-4170-bf96-4562e3083069 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:18.149543 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b19869-26fe-4170-bf96-4562e3083069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:18.205036 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-39b19869-26fe-4170-bf96-4562e3083069 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:18.210414 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-39b19869-26fe-4170-bf96-4562e3083069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:18.225779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa279aae-c8c8-45fc-b4e5-c09c9c3e3542 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:18.227196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-fa279aae-c8c8-45fc-b4e5-c09c9c3e3542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:18.251925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 840/1653] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:18.255756 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b19869-26fe-4170-bf96-4562e3083069 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:18.283570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b19869-26fe-4170-bf96-4562e3083069 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:18.302583 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df763b-6bce-468d-bd05-29042c76839a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:18.303298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df763b-6bce-468d-bd05-29042c76839a admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:18.306061 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df763b-6bce-468d-bd05-29042c76839a admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:18.306061 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df763b-6bce-468d-bd05-29042c76839a admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:18.306061 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df763b-6bce-468d-bd05-29042c76839a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:12:18.306367 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df763b-6bce-468d-bd05-29042c76839a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:18.310358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df763b-6bce-468d-bd05-29042c76839a admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:18.333875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:18.346658 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 841/1654] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:12:18.348329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df763b-6bce-468d-bd05-29042c76839a admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:18.364984 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:18.371873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df763b-6bce-468d-bd05-29042c76839a admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:18.405089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b3ae8-b40a-4160-b5e0-30d4d36d0388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:18.405738 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b3ae8-b40a-4160-b5e0-30d4d36d0388 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:18.406022 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b3ae8-b40a-4160-b5e0-30d4d36d0388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:18.406283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b3ae8-b40a-4160-b5e0-30d4d36d0388 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:18.406905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b3ae8-b40a-4160-b5e0-30d4d36d0388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:18.408887 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b3ae8-b40a-4160-b5e0-30d4d36d0388 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:18.492869 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa279aae-c8c8-45fc-b4e5-c09c9c3e3542 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:18.499986 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d3783-7ef6-418d-a894-8c1731974490 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:18.501333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d3783-7ef6-418d-a894-8c1731974490 service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:18.501769 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d3783-7ef6-418d-a894-8c1731974490 service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:18.502186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d3783-7ef6-418d-a894-8c1731974490 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:18.512541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-177d3783-7ef6-418d-a894-8c1731974490 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:18.543026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 842/1655] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:18 2022] POST /identity/v3/projects => generated 403 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:18.547234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b3ae8-b40a-4160-b5e0-30d4d36d0388 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:18.559084 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d3783-7ef6-418d-a894-8c1731974490 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:18.565316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d3783-7ef6-418d-a894-8c1731974490 service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:18.599534 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 814/1656] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:18 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:18.705946 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b3ae8-b40a-4160-b5e0-30d4d36d0388 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:18.719729 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e50b3-8bb9-411d-8466-87ee6781074b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:18.720917 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e50b3-8bb9-411d-8466-87ee6781074b admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:18.721384 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e50b3-8bb9-411d-8466-87ee6781074b admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:18.721919 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e50b3-8bb9-411d-8466-87ee6781074b admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:18.722734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e50b3-8bb9-411d-8466-87ee6781074b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:18.726579 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e50b3-8bb9-411d-8466-87ee6781074b admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:18.768921 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 843/1657] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:18 2022] POST /identity/v3/users => generated 302 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:12:18.772905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d3783-7ef6-418d-a894-8c1731974490 service nova] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:18.801689 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d3783-7ef6-418d-a894-8c1731974490 service nova] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:18.821538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79108e-8d72-486a-9192-04444abaf6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:18.822598 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79108e-8d72-486a-9192-04444abaf6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:18.823226 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79108e-8d72-486a-9192-04444abaf6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:18.824082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79108e-8d72-486a-9192-04444abaf6a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:18.825481 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79108e-8d72-486a-9192-04444abaf6a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:18.827134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79108e-8d72-486a-9192-04444abaf6a0 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:18.856097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 815/1658] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:18.859767 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e50b3-8bb9-411d-8466-87ee6781074b admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:18.875792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e50b3-8bb9-411d-8466-87ee6781074b admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:18.888863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc35dd-9918-4766-8aa7-5d5801ce58aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:18.890322 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc35dd-9918-4766-8aa7-5d5801ce58aa admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:18.890872 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc35dd-9918-4766-8aa7-5d5801ce58aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:18.891498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc35dd-9918-4766-8aa7-5d5801ce58aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:18.892719 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc35dd-9918-4766-8aa7-5d5801ce58aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:18.895250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc35dd-9918-4766-8aa7-5d5801ce58aa admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:18.909473 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 844/1659] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:12:18.916631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79108e-8d72-486a-9192-04444abaf6a0 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:18.949864 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79108e-8d72-486a-9192-04444abaf6a0 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:18.963491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a1853-66c8-4d82-b277-50889dc205f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:18.965411 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a1853-66c8-4d82-b277-50889dc205f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:18.966801 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a1853-66c8-4d82-b277-50889dc205f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:18.968409 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a1853-66c8-4d82-b277-50889dc205f8 admin admin] PATH_INFO: `/v3/projects/c85912dcdc424208a709c4cc4959ae63/users/9cda314fd3604be6b37669a2524c0e3c/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:19.014781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a1853-66c8-4d82-b277-50889dc205f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c85912dcdc424208a709c4cc4959ae63, user_id=9cda314fd3604be6b37669a2524c0e3c, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:19.017662 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a1853-66c8-4d82-b277-50889dc205f8 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:19.049102 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 816/1660] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:18 2022] PUT /identity/v3/projects/c85912dcdc424208a709c4cc4959ae63/users/9cda314fd3604be6b37669a2524c0e3c/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:19.054358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09af728-6d87-48e7-b5d6-0f823b5984fc None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:19.054696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09af728-6d87-48e7-b5d6-0f823b5984fc None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:19.055279 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09af728-6d87-48e7-b5d6-0f823b5984fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:19.115743 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-e09af728-6d87-48e7-b5d6-0f823b5984fc None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:19.117213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e09af728-6d87-48e7-b5d6-0f823b5984fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:19.279773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 845/1661] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:19.314212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a1853-66c8-4d82-b277-50889dc205f8 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:19.316885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c86a1853-66c8-4d82-b277-50889dc205f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:19.471683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a1853-66c8-4d82-b277-50889dc205f8 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:19.476587 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c8d31-cb64-4d41-ac0d-d6205f0a8089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:19.477368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c8d31-cb64-4d41-ac0d-d6205f0a8089 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:19.477713 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c8d31-cb64-4d41-ac0d-d6205f0a8089 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:19.478116 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c8d31-cb64-4d41-ac0d-d6205f0a8089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:19.492272 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c8d31-cb64-4d41-ac0d-d6205f0a8089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:19.494510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c8d31-cb64-4d41-ac0d-d6205f0a8089 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:19.514054 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 817/1662] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:19.644508 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981a5ef-9a2c-4e75-b2ff-d697d1acdc5e None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:19.644966 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981a5ef-9a2c-4e75-b2ff-d697d1acdc5e None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:19.645383 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981a5ef-9a2c-4e75-b2ff-d697d1acdc5e None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:19.647386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 846/1663] 10.222.0.17 () {54 vars in 912 bytes} [Tue Aug 23 07:12:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:12:19.654455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b121e-d710-4380-b02c-42b60a66342b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:19.654867 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b121e-d710-4380-b02c-42b60a66342b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:19.655314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b121e-d710-4380-b02c-42b60a66342b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:19.759006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-b75b121e-d710-4380-b02c-42b60a66342b None None] MFA Rules not processed for user `938364c02d8d4dd2ae832c5129be3cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:19.765631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b75b121e-d710-4380-b02c-42b60a66342b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:19.785223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 818/1664] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 23 07:12:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:19.815238 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981a5ef-9a2c-4e75-b2ff-d697d1acdc5e None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:19.833934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981a5ef-9a2c-4e75-b2ff-d697d1acdc5e None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:19.843660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718cd91-5796-4714-b396-82876adebe3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:19.845580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718cd91-5796-4714-b396-82876adebe3a service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:19.846419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718cd91-5796-4714-b396-82876adebe3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:19.847796 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718cd91-5796-4714-b396-82876adebe3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:19.888010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718cd91-5796-4714-b396-82876adebe3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:19.893799 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718cd91-5796-4714-b396-82876adebe3a service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:19.926865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 847/1665] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:20.986924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef79726-1b8e-4d68-841e-dfbda2668f26 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:20.988923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef79726-1b8e-4d68-841e-dfbda2668f26 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:20.989651 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef79726-1b8e-4d68-841e-dfbda2668f26 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:20.991489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 819/1666] 10.222.0.17 () {54 vars in 961 bytes} [Tue Aug 23 07:12:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:12:21.000621 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cabae-4a78-4dff-900f-55d33956d0b8 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:21.002694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cabae-4a78-4dff-900f-55d33956d0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:21.006501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cabae-4a78-4dff-900f-55d33956d0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:21.114082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-004cabae-4a78-4dff-900f-55d33956d0b8 None None] MFA Rules not processed for user `a9b0f54c988e43bda474f978a69fa4fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:21.125754 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-004cabae-4a78-4dff-900f-55d33956d0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:21.272155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:21.312447 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 848/1667] 10.222.0.17 () {58 vars in 1058 bytes} [Tue Aug 23 07:12:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:21.319893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef79726-1b8e-4d68-841e-dfbda2668f26 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:21.360443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef79726-1b8e-4d68-841e-dfbda2668f26 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:21.396009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72775e-d7d5-454c-8527-e3b7ae1fe528 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9b0f54c988e43bda474f978a69fa4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:21.396913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72775e-d7d5-454c-8527-e3b7ae1fe528 service placement] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:21.398906 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72775e-d7d5-454c-8527-e3b7ae1fe528 service placement] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:21.399188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72775e-d7d5-454c-8527-e3b7ae1fe528 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:21.410101 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3e72775e-d7d5-454c-8527-e3b7ae1fe528 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:21.474412 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72775e-d7d5-454c-8527-e3b7ae1fe528 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:21.475540 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b2daf-cffc-429b-9cbf-3ff84cfb263f None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:21.475997 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b2daf-cffc-429b-9cbf-3ff84cfb263f None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:21.476127 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72775e-d7d5-454c-8527-e3b7ae1fe528 service placement] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:21.476588 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b2daf-cffc-429b-9cbf-3ff84cfb263f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:21.504479 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 820/1668] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:12:21.516802 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-a78b2daf-cffc-429b-9cbf-3ff84cfb263f None None] MFA Rules not processed for user `64d54cb95b264c18b45d90822507fae1`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:21.518156 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a78b2daf-cffc-429b-9cbf-3ff84cfb263f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:21.607353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 849/1669] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:21.735942 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488e115-b0d9-4700-aca9-5f647a924d86 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:21.736227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488e115-b0d9-4700-aca9-5f647a924d86 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:21.736680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488e115-b0d9-4700-aca9-5f647a924d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:21.816983 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-e488e115-b0d9-4700-aca9-5f647a924d86 None None] MFA Rules not processed for user `9cda314fd3604be6b37669a2524c0e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:21.825938 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e488e115-b0d9-4700-aca9-5f647a924d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:21.900993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b54981-bb9c-42dd-9eaa-b23f3640aed6 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:21.901281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b54981-bb9c-42dd-9eaa-b23f3640aed6 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:21.901558 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b54981-bb9c-42dd-9eaa-b23f3640aed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:21.972709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:22.012401 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-44b54981-bb9c-42dd-9eaa-b23f3640aed6 None None] MFA Rules not processed for user `6cbcc9668f454c0e8c27495a1827d703`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:22.016095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-44b54981-bb9c-42dd-9eaa-b23f3640aed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:22.050117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 821/1670] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:21 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:22.062038 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488e115-b0d9-4700-aca9-5f647a924d86 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:22.112418 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488e115-b0d9-4700-aca9-5f647a924d86 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:22.132719 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5518b8-55cd-4631-88c3-c84f9f37761a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:22.136714 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5518b8-55cd-4631-88c3-c84f9f37761a admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:22.137004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5518b8-55cd-4631-88c3-c84f9f37761a admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:22.138463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5518b8-55cd-4631-88c3-c84f9f37761a admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:22.139149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5518b8-55cd-4631-88c3-c84f9f37761a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:22.151570 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5518b8-55cd-4631-88c3-c84f9f37761a admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:22.270383 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 850/1671] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:22.270383 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b54981-bb9c-42dd-9eaa-b23f3640aed6 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:22.275518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 822/1672] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:22 2022] POST /identity/v3/projects => generated 405 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:22.275795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-44b54981-bb9c-42dd-9eaa-b23f3640aed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:22.276728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5518b8-55cd-4631-88c3-c84f9f37761a admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:22.336314 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b54981-bb9c-42dd-9eaa-b23f3640aed6 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:22.394231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5518b8-55cd-4631-88c3-c84f9f37761a admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:22.402480 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929efdf-1c7b-418b-8a20-60b988f3cf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:22.404350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929efdf-1c7b-418b-8a20-60b988f3cf4a admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:22.404637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929efdf-1c7b-418b-8a20-60b988f3cf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:22.404915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929efdf-1c7b-418b-8a20-60b988f3cf4a admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:22.405563 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929efdf-1c7b-418b-8a20-60b988f3cf4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:22.411301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929efdf-1c7b-418b-8a20-60b988f3cf4a admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:22.518500 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab96df8-fe71-4ca7-a13b-124c6dd20d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:22.519154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab96df8-fe71-4ca7-a13b-124c6dd20d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:22.521301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 823/1673] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:22 2022] POST /identity/v3/users => generated 309 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:22.522987 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d03a34-03b5-4ead-beff-4b5513d90073 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:22.523553 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab96df8-fe71-4ca7-a13b-124c6dd20d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:22.524060 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab96df8-fe71-4ca7-a13b-124c6dd20d15 admin admin] PATH_INFO: `/v3/users/326bc6f8519c4497b1a7d7caeb6b4a77` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:22.526079 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d03a34-03b5-4ead-beff-4b5513d90073 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:22.526937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab96df8-fe71-4ca7-a13b-124c6dd20d15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=326bc6f8519c4497b1a7d7caeb6b4a77)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:22.527386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d03a34-03b5-4ead-beff-4b5513d90073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:22.533126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab96df8-fe71-4ca7-a13b-124c6dd20d15 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:22.563721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:22.573109 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-93d03a34-03b5-4ead-beff-4b5513d90073 None None] MFA Rules not processed for user `6cbcc9668f454c0e8c27495a1827d703`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:22.577791 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-93d03a34-03b5-4ead-beff-4b5513d90073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:22.631501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fab96df8-fe71-4ca7-a13b-124c6dd20d15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '326bc6f8519c4497b1a7d7caeb6b4a77'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:22.632195 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fab96df8-fe71-4ca7-a13b-124c6dd20d15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '326bc6f8519c4497b1a7d7caeb6b4a77'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:22.632826 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fab96df8-fe71-4ca7-a13b-124c6dd20d15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '326bc6f8519c4497b1a7d7caeb6b4a77'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:22.645529 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fab96df8-fe71-4ca7-a13b-124c6dd20d15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '326bc6f8519c4497b1a7d7caeb6b4a77'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:22.687776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fab96df8-fe71-4ca7-a13b-124c6dd20d15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '326bc6f8519c4497b1a7d7caeb6b4a77'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:22.722836 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 851/1674] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:12:22 2022] DELETE /identity/v3/users/326bc6f8519c4497b1a7d7caeb6b4a77 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:22.726758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cdc26-7a0c-4e2a-a4b5-705555fdbc39 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:22.726895 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 824/1675] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:22.726895 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d03a34-03b5-4ead-beff-4b5513d90073 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:22.730305 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cdc26-7a0c-4e2a-a4b5-705555fdbc39 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:22.730305 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cdc26-7a0c-4e2a-a4b5-705555fdbc39 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:22.730728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-93d03a34-03b5-4ead-beff-4b5513d90073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:22.738119 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 852/1676] 10.222.0.17 () {54 vars in 912 bytes} [Tue Aug 23 07:12:22 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:12:22.741322 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede7c76-07dc-4e7e-b842-76b8716d56cf None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:22.745425 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede7c76-07dc-4e7e-b842-76b8716d56cf None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:22.746020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede7c76-07dc-4e7e-b842-76b8716d56cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:22.754071 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d03a34-03b5-4ead-beff-4b5513d90073 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:22.793082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-3ede7c76-07dc-4e7e-b842-76b8716d56cf None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:22.797796 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3ede7c76-07dc-4e7e-b842-76b8716d56cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:22.892024 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b62d9-5a36-43ec-9580-a7df6088849c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:22.892847 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b62d9-5a36-43ec-9580-a7df6088849c admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:22.893412 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b62d9-5a36-43ec-9580-a7df6088849c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:22.893830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b62d9-5a36-43ec-9580-a7df6088849c admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:22.894643 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b62d9-5a36-43ec-9580-a7df6088849c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:22.896725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b62d9-5a36-43ec-9580-a7df6088849c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:22.898848 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:22.918988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 825/1677] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:22.926081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7e1c0-0845-4373-9d2f-04a7c12efde2 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:22.926478 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7e1c0-0845-4373-9d2f-04a7c12efde2 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:22.926743 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7e1c0-0845-4373-9d2f-04a7c12efde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:22.967591 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-b3a7e1c0-0845-4373-9d2f-04a7c12efde2 None None] MFA Rules not processed for user `938364c02d8d4dd2ae832c5129be3cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:22.971188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b3a7e1c0-0845-4373-9d2f-04a7c12efde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:22.971770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 853/1678] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:22.973249 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede7c76-07dc-4e7e-b842-76b8716d56cf None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:22.975298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3ede7c76-07dc-4e7e-b842-76b8716d56cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:23.144174 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede7c76-07dc-4e7e-b842-76b8716d56cf None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:23.159212 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759bb41-0def-4bbd-91b4-36bdc061b4b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:23.160383 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759bb41-0def-4bbd-91b4-36bdc061b4b7 service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:23.160665 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759bb41-0def-4bbd-91b4-36bdc061b4b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:23.161483 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759bb41-0def-4bbd-91b4-36bdc061b4b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:23.164902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 826/1679] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 23 07:12:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:23.166126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c759bb41-0def-4bbd-91b4-36bdc061b4b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:23.166251 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7e1c0-0845-4373-9d2f-04a7c12efde2 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:23.184011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7e1c0-0845-4373-9d2f-04a7c12efde2 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:23.191938 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772a323-14a2-4921-aaeb-4f536f8bef55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:23.195925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772a323-14a2-4921-aaeb-4f536f8bef55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:23.196219 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772a323-14a2-4921-aaeb-4f536f8bef55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:23.196505 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772a323-14a2-4921-aaeb-4f536f8bef55 admin admin] PATH_INFO: `/v3/projects/7aeba7cc278d437b91761334473f2ac2/users/6655cece7a214bc0b11d61015444e6a5/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:23.204933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:23.281192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6772a323-14a2-4921-aaeb-4f536f8bef55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aeba7cc278d437b91761334473f2ac2, user_id=6655cece7a214bc0b11d61015444e6a5, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:23.283357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759bb41-0def-4bbd-91b4-36bdc061b4b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:23.285331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759bb41-0def-4bbd-91b4-36bdc061b4b7 service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:23.297666 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6772a323-14a2-4921-aaeb-4f536f8bef55 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:23.305162 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 854/1680] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:23.318892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759bb41-0def-4bbd-91b4-36bdc061b4b7 service nova] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:23.322824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c759bb41-0def-4bbd-91b4-36bdc061b4b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:23.331128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 827/1681] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:23 2022] PUT /identity/v3/projects/7aeba7cc278d437b91761334473f2ac2/users/6655cece7a214bc0b11d61015444e6a5/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:23.331128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772a323-14a2-4921-aaeb-4f536f8bef55 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:23.354706 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759bb41-0def-4bbd-91b4-36bdc061b4b7 service nova] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:23.440843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772a323-14a2-4921-aaeb-4f536f8bef55 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:23.446552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75b1ed-a586-48dd-a758-8277e6b9232c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:23.447228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75b1ed-a586-48dd-a758-8277e6b9232c service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:23.447674 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75b1ed-a586-48dd-a758-8277e6b9232c service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:23.448168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75b1ed-a586-48dd-a758-8277e6b9232c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:23.460587 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75b1ed-a586-48dd-a758-8277e6b9232c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:23.464260 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75b1ed-a586-48dd-a758-8277e6b9232c service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:23.475551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 828/1682] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:23.476514 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75b1ed-a586-48dd-a758-8277e6b9232c service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:23.512470 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7bbac-57d4-4574-bb99-aa9f5aeb51f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:23.513105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7bbac-57d4-4574-bb99-aa9f5aeb51f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:23.513384 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7bbac-57d4-4574-bb99-aa9f5aeb51f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:23.513971 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7bbac-57d4-4574-bb99-aa9f5aeb51f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:23.613428 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75b1ed-a586-48dd-a758-8277e6b9232c service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:23.621332 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf539711-5efc-4759-9b79-ecead5aa07d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:23.623620 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf539711-5efc-4759-9b79-ecead5aa07d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:23.624082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf539711-5efc-4759-9b79-ecead5aa07d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:23.624495 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf539711-5efc-4759-9b79-ecead5aa07d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:23.625375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf539711-5efc-4759-9b79-ecead5aa07d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:23.628356 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf539711-5efc-4759-9b79-ecead5aa07d8 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:23.645573 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 829/1683] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:23.652012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf539711-5efc-4759-9b79-ecead5aa07d8 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:23.675816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf539711-5efc-4759-9b79-ecead5aa07d8 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:23.685109 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7bbac-57d4-4574-bb99-aa9f5aeb51f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:23.689359 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7bbac-57d4-4574-bb99-aa9f5aeb51f8 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:23.692010 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153b071-ecd3-4e10-b184-39ff7254ac31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:23.692959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153b071-ecd3-4e10-b184-39ff7254ac31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:23.693315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153b071-ecd3-4e10-b184-39ff7254ac31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:23.693703 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153b071-ecd3-4e10-b184-39ff7254ac31 admin admin] PATH_INFO: `/v3/projects/7aeba7cc278d437b91761334473f2ac2/users/6655cece7a214bc0b11d61015444e6a5/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:23.698639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153b071-ecd3-4e10-b184-39ff7254ac31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aeba7cc278d437b91761334473f2ac2, user_id=6655cece7a214bc0b11d61015444e6a5, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:23.704288 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153b071-ecd3-4e10-b184-39ff7254ac31 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:23.717499 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 855/1684] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:23.736689 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 830/1685] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:23 2022] PUT /identity/v3/projects/7aeba7cc278d437b91761334473f2ac2/users/6655cece7a214bc0b11d61015444e6a5/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:25.479181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7bbac-57d4-4574-bb99-aa9f5aeb51f8 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:25.480825 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-1cb7bbac-57d4-4574-bb99-aa9f5aeb51f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:25.507680 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7bbac-57d4-4574-bb99-aa9f5aeb51f8 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:25.585583 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f13cb-0619-40f3-a457-c1b0ad48e3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:25.586934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f13cb-0619-40f3-a457-c1b0ad48e3f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:25.587214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f13cb-0619-40f3-a457-c1b0ad48e3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:25.589051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f13cb-0619-40f3-a457-c1b0ad48e3f1 admin admin] PATH_INFO: `/v3/projects/f1dc52471ac841d5be9887ddfd1503c1` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:25.591914 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f13cb-0619-40f3-a457-c1b0ad48e3f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1dc52471ac841d5be9887ddfd1503c1)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:25.597451 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f13cb-0619-40f3-a457-c1b0ad48e3f1 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:25.640100 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:25.687663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-965f13cb-0619-40f3-a457-c1b0ad48e3f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1dc52471ac841d5be9887ddfd1503c1', 'request_id': 'req-965f13cb-0619-40f3-a457-c1b0ad48e3f1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:25.695646 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-965f13cb-0619-40f3-a457-c1b0ad48e3f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1dc52471ac841d5be9887ddfd1503c1', 'request_id': 'req-965f13cb-0619-40f3-a457-c1b0ad48e3f1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:25.707947 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-965f13cb-0619-40f3-a457-c1b0ad48e3f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1dc52471ac841d5be9887ddfd1503c1', 'request_id': 'req-965f13cb-0619-40f3-a457-c1b0ad48e3f1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:25.726121 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-965f13cb-0619-40f3-a457-c1b0ad48e3f1 admin admin] The token cache is being invalidate because project f1dc52471ac841d5be9887ddfd1503c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:12:25.727035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-965f13cb-0619-40f3-a457-c1b0ad48e3f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:25.728317 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-965f13cb-0619-40f3-a457-c1b0ad48e3f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:25.734193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 856/1686] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:12:25 2022] DELETE /identity/v3/projects/f1dc52471ac841d5be9887ddfd1503c1 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:25.736184 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2166da-18e6-48db-8270-d2ed12a84750 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:25.737986 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2166da-18e6-48db-8270-d2ed12a84750 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:25.738511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2166da-18e6-48db-8270-d2ed12a84750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:25.793134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:25.814998 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-8a2166da-18e6-48db-8270-d2ed12a84750 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:25.824782 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67813cdc-68a7-42b7-942f-230ff6e4d93c None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:25.825664 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67813cdc-68a7-42b7-942f-230ff6e4d93c None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:25.827211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67813cdc-68a7-42b7-942f-230ff6e4d93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:25.835765 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8a2166da-18e6-48db-8270-d2ed12a84750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:25.979070 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-67813cdc-68a7-42b7-942f-230ff6e4d93c None None] MFA Rules not processed for user `471dc5c1db64411ab1d80ad6c038e519`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:25.986243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-67813cdc-68a7-42b7-942f-230ff6e4d93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:26.054680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 831/1687] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:26.060414 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21deb1fc-450d-40e5-b0f3-34f988e23cbb None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:26.060729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21deb1fc-450d-40e5-b0f3-34f988e23cbb None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:26.063289 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21deb1fc-450d-40e5-b0f3-34f988e23cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:26.206459 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-21deb1fc-450d-40e5-b0f3-34f988e23cbb None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:26.219692 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-21deb1fc-450d-40e5-b0f3-34f988e23cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:26.258620 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 832/1688] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:26.263634 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21deb1fc-450d-40e5-b0f3-34f988e23cbb None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:26.289893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21deb1fc-450d-40e5-b0f3-34f988e23cbb None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:26.309705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ded5d0-4635-4f11-a3a0-dfc8d6250792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:26.310211 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ded5d0-4635-4f11-a3a0-dfc8d6250792 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:26.310485 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ded5d0-4635-4f11-a3a0-dfc8d6250792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:26.310752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ded5d0-4635-4f11-a3a0-dfc8d6250792 admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:26.311431 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ded5d0-4635-4f11-a3a0-dfc8d6250792 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:12:26.311733 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ded5d0-4635-4f11-a3a0-dfc8d6250792 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:26.318353 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ded5d0-4635-4f11-a3a0-dfc8d6250792 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:26.373355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 833/1689] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:12:26.376523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532fd68-5d61-493d-851d-e0d3db8a27c8 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:26.379525 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 857/1690] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 560 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:26.379658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532fd68-5d61-493d-851d-e0d3db8a27c8 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:26.380462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532fd68-5d61-493d-851d-e0d3db8a27c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:26.381557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67813cdc-68a7-42b7-942f-230ff6e4d93c None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:26.425150 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67813cdc-68a7-42b7-942f-230ff6e4d93c None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:26.433510 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fdb90a-6e83-49a0-8d9e-dc785d4f88a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:26.434908 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdb90a-6e83-49a0-8d9e-dc785d4f88a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:26.435197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdb90a-6e83-49a0-8d9e-dc785d4f88a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:26.435584 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdb90a-6e83-49a0-8d9e-dc785d4f88a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:26.436234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fdb90a-6e83-49a0-8d9e-dc785d4f88a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:26.438357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fdb90a-6e83-49a0-8d9e-dc785d4f88a8 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:26.478608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-7532fd68-5d61-493d-851d-e0d3db8a27c8 None None] MFA Rules not processed for user `6655cece7a214bc0b11d61015444e6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:26.483527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-7532fd68-5d61-493d-851d-e0d3db8a27c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:26.507148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 858/1691] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:26 2022] POST /identity/v3/projects => generated 367 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:26.507148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fdb90a-6e83-49a0-8d9e-dc785d4f88a8 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:26.508488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c6fdb90a-6e83-49a0-8d9e-dc785d4f88a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:26.812880 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fdb90a-6e83-49a0-8d9e-dc785d4f88a8 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:26.832770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7950b7-6ed1-4905-8f41-da2b78362a3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:26.832970 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 834/1692] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:26 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:26.832970 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532fd68-5d61-493d-851d-e0d3db8a27c8 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:26.839212 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7950b7-6ed1-4905-8f41-da2b78362a3e service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:26.839436 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7950b7-6ed1-4905-8f41-da2b78362a3e service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:26.842619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7950b7-6ed1-4905-8f41-da2b78362a3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:26.903167 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:27.068010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7950b7-6ed1-4905-8f41-da2b78362a3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:27.073812 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7950b7-6ed1-4905-8f41-da2b78362a3e service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:27.077416 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532fd68-5d61-493d-851d-e0d3db8a27c8 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:27.099188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5114b-5397-4f6e-9f65-b9feb5f44975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:27.104052 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5114b-5397-4f6e-9f65-b9feb5f44975 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:27.105050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5114b-5397-4f6e-9f65-b9feb5f44975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:27.105697 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5114b-5397-4f6e-9f65-b9feb5f44975 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:27.106379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5114b-5397-4f6e-9f65-b9feb5f44975 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:27.115927 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5114b-5397-4f6e-9f65-b9feb5f44975 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:27.117635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 859/1693] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:27.206413 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7950b7-6ed1-4905-8f41-da2b78362a3e service nova] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:27.207886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3a7950b7-6ed1-4905-8f41-da2b78362a3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:27.223823 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 835/1694] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:26 2022] POST /identity/v3/users => generated 284 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:27.224852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5114b-5397-4f6e-9f65-b9feb5f44975 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:27.238830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:27.253472 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5114b-5397-4f6e-9f65-b9feb5f44975 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:27.262851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7950b7-6ed1-4905-8f41-da2b78362a3e service nova] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:27.288035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46dd4f2-f645-492c-b06c-4e50cb70105f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:27.288608 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46dd4f2-f645-492c-b06c-4e50cb70105f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:27.297420 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46dd4f2-f645-492c-b06c-4e50cb70105f admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:27.297772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46dd4f2-f645-492c-b06c-4e50cb70105f admin admin] PATH_INFO: `/v3/users/4cf8918a05f8452a9bb8e17a47de7a8a` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:27.299288 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9525c25-a4e8-47d3-8d85-5c8d8eef8f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:27.299976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9525c25-a4e8-47d3-8d85-5c8d8eef8f52 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:27.300258 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9525c25-a4e8-47d3-8d85-5c8d8eef8f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:27.300532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9525c25-a4e8-47d3-8d85-5c8d8eef8f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:27.301190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9525c25-a4e8-47d3-8d85-5c8d8eef8f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:27.308869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46dd4f2-f645-492c-b06c-4e50cb70105f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cf8918a05f8452a9bb8e17a47de7a8a)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:27.311097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9525c25-a4e8-47d3-8d85-5c8d8eef8f52 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:27.319536 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46dd4f2-f645-492c-b06c-4e50cb70105f admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:27.369916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 836/1695] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:27.371774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9525c25-a4e8-47d3-8d85-5c8d8eef8f52 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:27.405553 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9525c25-a4e8-47d3-8d85-5c8d8eef8f52 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:27.419499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c644d9-1a18-4c86-becc-53772ea4dce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:27.420589 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c644d9-1a18-4c86-becc-53772ea4dce6 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:27.421093 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c644d9-1a18-4c86-becc-53772ea4dce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:27.421586 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c644d9-1a18-4c86-becc-53772ea4dce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:27.422455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c644d9-1a18-4c86-becc-53772ea4dce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:27.424348 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c644d9-1a18-4c86-becc-53772ea4dce6 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:27.454824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 837/1696] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:27.460514 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c644d9-1a18-4c86-becc-53772ea4dce6 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:27.497549 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c644d9-1a18-4c86-becc-53772ea4dce6 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:27.516914 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb08cad-16e1-4f11-af9f-cf5521eeb491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:27.517995 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb08cad-16e1-4f11-af9f-cf5521eeb491 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:27.517995 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb08cad-16e1-4f11-af9f-cf5521eeb491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:27.518205 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb08cad-16e1-4f11-af9f-cf5521eeb491 admin admin] PATH_INFO: `/v3/projects/771b3076a9ac4c44819c559859a1add3/users/a0e300baac294ebe867407ff20334637/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:27.574514 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d46dd4f2-f645-492c-b06c-4e50cb70105f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cf8918a05f8452a9bb8e17a47de7a8a'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:27.579399 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d46dd4f2-f645-492c-b06c-4e50cb70105f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cf8918a05f8452a9bb8e17a47de7a8a'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:27.584606 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d46dd4f2-f645-492c-b06c-4e50cb70105f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cf8918a05f8452a9bb8e17a47de7a8a'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:27.606154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb08cad-16e1-4f11-af9f-cf5521eeb491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=771b3076a9ac4c44819c559859a1add3, user_id=a0e300baac294ebe867407ff20334637, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:27.618936 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb08cad-16e1-4f11-af9f-cf5521eeb491 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:27.626668 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d46dd4f2-f645-492c-b06c-4e50cb70105f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cf8918a05f8452a9bb8e17a47de7a8a'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:27.669856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 838/1697] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:27 2022] PUT /identity/v3/projects/771b3076a9ac4c44819c559859a1add3/users/a0e300baac294ebe867407ff20334637/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:27.671377 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5b849-f51c-4ae7-91c7-9b6fd8819251 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:27.674484 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5b849-f51c-4ae7-91c7-9b6fd8819251 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:27.675485 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5b849-f51c-4ae7-91c7-9b6fd8819251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:27.683596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d46dd4f2-f645-492c-b06c-4e50cb70105f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cf8918a05f8452a9bb8e17a47de7a8a'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:27.724185 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 860/1698] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:12:27 2022] DELETE /identity/v3/users/4cf8918a05f8452a9bb8e17a47de7a8a => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:12:27.725679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889edf2-8be5-41f4-89a7-455df0ffa091 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:27.729041 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889edf2-8be5-41f4-89a7-455df0ffa091 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:27.729041 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889edf2-8be5-41f4-89a7-455df0ffa091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:27.826535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-8889edf2-8be5-41f4-89a7-455df0ffa091 None None] MFA Rules not processed for user `a0e300baac294ebe867407ff20334637`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:27.834615 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8889edf2-8be5-41f4-89a7-455df0ffa091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:27.851439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-ccd5b849-f51c-4ae7-91c7-9b6fd8819251 None None] MFA Rules not processed for user `b3f1ae88e29f4172a7db7393d517a893`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:27.854640 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ccd5b849-f51c-4ae7-91c7-9b6fd8819251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:27.996410 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 861/1699] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:27.998464 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182d01c-c038-483f-b03f-9d5f4d7e8912 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:27.998937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182d01c-c038-483f-b03f-9d5f4d7e8912 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:28.002543 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182d01c-c038-483f-b03f-9d5f4d7e8912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:28.048960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-6182d01c-c038-483f-b03f-9d5f4d7e8912 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:28.050411 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6182d01c-c038-483f-b03f-9d5f4d7e8912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:28.074656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 839/1700] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 23 07:12:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:28.075861 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5b849-f51c-4ae7-91c7-9b6fd8819251 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:28.079794 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ccd5b849-f51c-4ae7-91c7-9b6fd8819251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:28.091940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:28.295674 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 862/1701] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:28.337128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5b849-f51c-4ae7-91c7-9b6fd8819251 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:28.363704 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182d01c-c038-483f-b03f-9d5f4d7e8912 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:28.365381 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393efd83-e9cc-4bd3-84e3-50c9c67aa555 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:28.366485 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393efd83-e9cc-4bd3-84e3-50c9c67aa555 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:28.366763 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393efd83-e9cc-4bd3-84e3-50c9c67aa555 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:28.368627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393efd83-e9cc-4bd3-84e3-50c9c67aa555 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:28.375138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6182d01c-c038-483f-b03f-9d5f4d7e8912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:28.431575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393efd83-e9cc-4bd3-84e3-50c9c67aa555 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:28.443187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393efd83-e9cc-4bd3-84e3-50c9c67aa555 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:28.492532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 840/1702] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:28.498419 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393efd83-e9cc-4bd3-84e3-50c9c67aa555 service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:28.504443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-393efd83-e9cc-4bd3-84e3-50c9c67aa555 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:28.568508 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182d01c-c038-483f-b03f-9d5f4d7e8912 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:28.603558 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c0f13-7267-4a0a-9cd2-d148d8cc10b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:28.605400 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c0f13-7267-4a0a-9cd2-d148d8cc10b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:28.605942 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c0f13-7267-4a0a-9cd2-d148d8cc10b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:28.606430 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c0f13-7267-4a0a-9cd2-d148d8cc10b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:28.622394 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c0f13-7267-4a0a-9cd2-d148d8cc10b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:28.624394 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c0f13-7267-4a0a-9cd2-d148d8cc10b8 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:28.649502 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 863/1703] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:12:28.711046 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:28.902069 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393efd83-e9cc-4bd3-84e3-50c9c67aa555 service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:28.918872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb26ab-6b6c-43a0-b978-92a5ddb62a05 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd88ec8b477443af95bff9af03471adf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:28.921178 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb26ab-6b6c-43a0-b978-92a5ddb62a05 service swift] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:28.922755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb26ab-6b6c-43a0-b978-92a5ddb62a05 service swift] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:28.923847 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb26ab-6b6c-43a0-b978-92a5ddb62a05 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:28.956157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cb26ab-6b6c-43a0-b978-92a5ddb62a05 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:28.962418 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cb26ab-6b6c-43a0-b978-92a5ddb62a05 service swift] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:28.990905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 841/1704] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 23 07:12:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 07:12:31.741484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c0f13-7267-4a0a-9cd2-d148d8cc10b8 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:31.742999 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d86c0f13-7267-4a0a-9cd2-d148d8cc10b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:31.767855 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c0f13-7267-4a0a-9cd2-d148d8cc10b8 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:31.775524 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f5e80-c4a7-48de-8963-38c959cb45f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:31.776195 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f5e80-c4a7-48de-8963-38c959cb45f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:31.776475 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f5e80-c4a7-48de-8963-38c959cb45f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:31.776920 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f5e80-c4a7-48de-8963-38c959cb45f7 admin admin] PATH_INFO: `/v3/projects/2877fe45fc8a4710a294486a9c1991b8` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:31.781701 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8b4c1-8948-4476-9c01-68bc507fbc8c None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:31.782691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8b4c1-8948-4476-9c01-68bc507fbc8c None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:31.783204 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8b4c1-8948-4476-9c01-68bc507fbc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:31.790409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f5e80-c4a7-48de-8963-38c959cb45f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2877fe45fc8a4710a294486a9c1991b8)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:31.792515 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f5e80-c4a7-48de-8963-38c959cb45f7 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:31.826584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-9fd8b4c1-8948-4476-9c01-68bc507fbc8c None None] MFA Rules not processed for user `938364c02d8d4dd2ae832c5129be3cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:31.835848 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9fd8b4c1-8948-4476-9c01-68bc507fbc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:31.863675 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 842/1705] 10.222.0.20 () {58 vars in 1015 bytes} [Tue Aug 23 07:12:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:31.943991 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8b4c1-8948-4476-9c01-68bc507fbc8c None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:31.957298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8b4c1-8948-4476-9c01-68bc507fbc8c None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:31.961841 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-b24f5e80-c4a7-48de-8963-38c959cb45f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2877fe45fc8a4710a294486a9c1991b8', 'request_id': 'req-b24f5e80-c4a7-48de-8963-38c959cb45f7'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:31.988579 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-b24f5e80-c4a7-48de-8963-38c959cb45f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2877fe45fc8a4710a294486a9c1991b8', 'request_id': 'req-b24f5e80-c4a7-48de-8963-38c959cb45f7'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:32.016087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-b24f5e80-c4a7-48de-8963-38c959cb45f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2877fe45fc8a4710a294486a9c1991b8', 'request_id': 'req-b24f5e80-c4a7-48de-8963-38c959cb45f7'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:32.058036 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-b24f5e80-c4a7-48de-8963-38c959cb45f7 admin admin] The token cache is being invalidate because project 2877fe45fc8a4710a294486a9c1991b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:12:32.059644 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-b24f5e80-c4a7-48de-8963-38c959cb45f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:32.060957 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-b24f5e80-c4a7-48de-8963-38c959cb45f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:32.064371 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 864/1706] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:12:31 2022] DELETE /identity/v3/projects/2877fe45fc8a4710a294486a9c1991b8 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:12:32.075461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf5fbd-6df8-4ee0-96db-dfd4f1d44420 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:32.075461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf5fbd-6df8-4ee0-96db-dfd4f1d44420 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:32.075461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf5fbd-6df8-4ee0-96db-dfd4f1d44420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:32.129151 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-bacf5fbd-6df8-4ee0-96db-dfd4f1d44420 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:32.138771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-bacf5fbd-6df8-4ee0-96db-dfd4f1d44420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:32.139735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77c6aa-0b1e-4b29-8a36-2ed99dcd83cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:32.140372 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77c6aa-0b1e-4b29-8a36-2ed99dcd83cf service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:32.140648 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77c6aa-0b1e-4b29-8a36-2ed99dcd83cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:32.140924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77c6aa-0b1e-4b29-8a36-2ed99dcd83cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:32.144126 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8a77c6aa-0b1e-4b29-8a36-2ed99dcd83cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:32.163924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77c6aa-0b1e-4b29-8a36-2ed99dcd83cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:32.166050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77c6aa-0b1e-4b29-8a36-2ed99dcd83cf service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:32.183324 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 843/1707] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:32.199587 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33940f0-8046-4e19-8398-62e2b6c9112c None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:32.200220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33940f0-8046-4e19-8398-62e2b6c9112c None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:32.200710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33940f0-8046-4e19-8398-62e2b6c9112c None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:32.203018 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 844/1708] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 23 07:12:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:12:32.276264 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eaa0ff-9dda-473c-9727-18f0b0b39ba9 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:32.276713 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eaa0ff-9dda-473c-9727-18f0b0b39ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:32.276891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eaa0ff-9dda-473c-9727-18f0b0b39ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:32.286888 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:32.369642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 865/1709] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:32.377716 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f4fd4-1e34-4084-9e90-09440f3ce7f7 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:32.378012 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f4fd4-1e34-4084-9e90-09440f3ce7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:32.378262 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f4fd4-1e34-4084-9e90-09440f3ce7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:32.389268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-78eaa0ff-9dda-473c-9727-18f0b0b39ba9 None None] MFA Rules not processed for user `a9b0f54c988e43bda474f978a69fa4fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:32.393350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-78eaa0ff-9dda-473c-9727-18f0b0b39ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:32.491828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-5d9f4fd4-1e34-4084-9e90-09440f3ce7f7 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:32.498134 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-5d9f4fd4-1e34-4084-9e90-09440f3ce7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:32.547527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 866/1710] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:32.551298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f4fd4-1e34-4084-9e90-09440f3ce7f7 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:32.584130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f4fd4-1e34-4084-9e90-09440f3ce7f7 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:32.610101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514397e-96e9-4bb8-9ce9-542ee499e024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:32.620828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514397e-96e9-4bb8-9ce9-542ee499e024 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:32.621601 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514397e-96e9-4bb8-9ce9-542ee499e024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:32.625681 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514397e-96e9-4bb8-9ce9-542ee499e024 admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:32.626881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514397e-96e9-4bb8-9ce9-542ee499e024 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:12:32.627636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514397e-96e9-4bb8-9ce9-542ee499e024 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:32.630772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514397e-96e9-4bb8-9ce9-542ee499e024 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:32.687446 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 867/1711] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:12:32.690552 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514397e-96e9-4bb8-9ce9-542ee499e024 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:32.691899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 845/1712] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 23 07:12:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:12:32.733894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514397e-96e9-4bb8-9ce9-542ee499e024 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:32.759764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eaa0ff-9dda-473c-9727-18f0b0b39ba9 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:32.760468 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153be05-238e-46b1-a185-cd6a044ceaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:32.761374 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153be05-238e-46b1-a185-cd6a044ceaa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:32.761657 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153be05-238e-46b1-a185-cd6a044ceaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:32.761890 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-78eaa0ff-9dda-473c-9727-18f0b0b39ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:32.762002 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153be05-238e-46b1-a185-cd6a044ceaa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:32.762564 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9153be05-238e-46b1-a185-cd6a044ceaa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:32.764526 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9153be05-238e-46b1-a185-cd6a044ceaa2 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:32.825087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eaa0ff-9dda-473c-9727-18f0b0b39ba9 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:32.825899 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 868/1713] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:32 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:32.829266 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153be05-238e-46b1-a185-cd6a044ceaa2 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:32.970701 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:33.050210 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153be05-238e-46b1-a185-cd6a044ceaa2 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:33.080457 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5707f-3b69-4bff-adb4-73b4b4990cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:33.081710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5707f-3b69-4bff-adb4-73b4b4990cb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:33.084982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5707f-3b69-4bff-adb4-73b4b4990cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:33.084982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5707f-3b69-4bff-adb4-73b4b4990cb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:33.086185 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5707f-3b69-4bff-adb4-73b4b4990cb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:33.101966 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5707f-3b69-4bff-adb4-73b4b4990cb4 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:33.110671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3a438-eb06-495a-98d4-38b16323afb1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9b0f54c988e43bda474f978a69fa4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:33.112316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3a438-eb06-495a-98d4-38b16323afb1 service placement] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:33.113152 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3a438-eb06-495a-98d4-38b16323afb1 service placement] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:33.114296 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3a438-eb06-495a-98d4-38b16323afb1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:33.164221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3a438-eb06-495a-98d4-38b16323afb1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:33.174107 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3a438-eb06-495a-98d4-38b16323afb1 service placement] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:33.193413 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 869/1714] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:32 2022] POST /identity/v3/users => generated 291 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:33.199502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 846/1715] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:33.209895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5707f-3b69-4bff-adb4-73b4b4990cb4 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:33.241904 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5707f-3b69-4bff-adb4-73b4b4990cb4 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:33.258941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256e45a-5f76-47ad-9e83-a719016196b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:33.260199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256e45a-5f76-47ad-9e83-a719016196b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:33.260490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256e45a-5f76-47ad-9e83-a719016196b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:33.260795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256e45a-5f76-47ad-9e83-a719016196b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:33.261400 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256e45a-5f76-47ad-9e83-a719016196b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:33.263040 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256e45a-5f76-47ad-9e83-a719016196b8 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:33.297171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 870/1716] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:33.299213 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3a438-eb06-495a-98d4-38b16323afb1 service placement] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:33.317249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3a438-eb06-495a-98d4-38b16323afb1 service placement] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:33.326596 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e105b72-dc16-42e4-b0fb-c388e4803303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:33.327735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e105b72-dc16-42e4-b0fb-c388e4803303 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:33.328158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e105b72-dc16-42e4-b0fb-c388e4803303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:33.328596 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e105b72-dc16-42e4-b0fb-c388e4803303 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:33.329396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e105b72-dc16-42e4-b0fb-c388e4803303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:33.332138 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e105b72-dc16-42e4-b0fb-c388e4803303 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:33.370882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 847/1717] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:33.374836 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256e45a-5f76-47ad-9e83-a719016196b8 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:33.410787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256e45a-5f76-47ad-9e83-a719016196b8 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:33.428703 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713641f1-0d0a-4993-8675-d90dbeae669d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:33.429393 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713641f1-0d0a-4993-8675-d90dbeae669d admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:33.429675 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713641f1-0d0a-4993-8675-d90dbeae669d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:33.429947 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713641f1-0d0a-4993-8675-d90dbeae669d admin admin] PATH_INFO: `/v3/projects/a6b8078cef194f7a9875eca797050074/users/2a5492ab69ee4afbb8c7b0a30818b680/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:33.504916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713641f1-0d0a-4993-8675-d90dbeae669d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6b8078cef194f7a9875eca797050074, user_id=2a5492ab69ee4afbb8c7b0a30818b680, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:33.508114 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713641f1-0d0a-4993-8675-d90dbeae669d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:33.566412 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 871/1718] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:33 2022] PUT /identity/v3/projects/a6b8078cef194f7a9875eca797050074/users/2a5492ab69ee4afbb8c7b0a30818b680/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:33.577911 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c156d2d-663f-4b07-bb73-38ce0eaf4f24 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:33.578199 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c156d2d-663f-4b07-bb73-38ce0eaf4f24 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:33.578480 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c156d2d-663f-4b07-bb73-38ce0eaf4f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:33.674568 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-2c156d2d-663f-4b07-bb73-38ce0eaf4f24 None None] MFA Rules not processed for user `2a5492ab69ee4afbb8c7b0a30818b680`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:33.676103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2c156d2d-663f-4b07-bb73-38ce0eaf4f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:33.952323 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 848/1719] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:33.956182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713641f1-0d0a-4993-8675-d90dbeae669d admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:34.141144 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:34.159480 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713641f1-0d0a-4993-8675-d90dbeae669d admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:34.172915 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25ac53-381c-4eae-ab82-5aa765c48c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:34.173662 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25ac53-381c-4eae-ab82-5aa765c48c1d admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:34.174006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25ac53-381c-4eae-ab82-5aa765c48c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:34.174328 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25ac53-381c-4eae-ab82-5aa765c48c1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:34.175030 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e25ac53-381c-4eae-ab82-5aa765c48c1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:34.177385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e25ac53-381c-4eae-ab82-5aa765c48c1d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:34.280270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c156d2d-663f-4b07-bb73-38ce0eaf4f24 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:34.293371 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 872/1720] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:33 2022] POST /identity/v3/projects => generated 379 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:34.302259 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c156d2d-663f-4b07-bb73-38ce0eaf4f24 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:34.552808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea9d9d-896f-4290-a8ef-293ca541f70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:34.553480 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea9d9d-896f-4290-a8ef-293ca541f70b admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:34.555590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea9d9d-896f-4290-a8ef-293ca541f70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:34.556087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea9d9d-896f-4290-a8ef-293ca541f70b admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:34.557519 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea9d9d-896f-4290-a8ef-293ca541f70b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:34.566658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea9d9d-896f-4290-a8ef-293ca541f70b admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:34.614821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 849/1721] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:34 2022] POST /identity/v3/users => generated 296 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:12:34.616207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25ac53-381c-4eae-ab82-5aa765c48c1d admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:34.637895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25ac53-381c-4eae-ab82-5aa765c48c1d admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:34.655682 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972aaa4-e02d-4caa-8928-3e046949aaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:34.658387 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972aaa4-e02d-4caa-8928-3e046949aaf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:34.659631 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972aaa4-e02d-4caa-8928-3e046949aaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:34.659920 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972aaa4-e02d-4caa-8928-3e046949aaf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:34.660609 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972aaa4-e02d-4caa-8928-3e046949aaf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:34.662249 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972aaa4-e02d-4caa-8928-3e046949aaf2 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:34.702988 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 873/1722] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:34.705072 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea9d9d-896f-4290-a8ef-293ca541f70b admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:34.726474 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea9d9d-896f-4290-a8ef-293ca541f70b admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:34.750718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2338e5-2d60-4f65-bc75-80bdeeed30d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:34.751443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2338e5-2d60-4f65-bc75-80bdeeed30d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:34.751734 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2338e5-2d60-4f65-bc75-80bdeeed30d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:34.752017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2338e5-2d60-4f65-bc75-80bdeeed30d3 admin admin] PATH_INFO: `/v3/projects/17b127e5f60a4b5ba93243a42b6f9c29/users/d4ec25c162eb481b846b60a41683e7c5/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:34.820144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2338e5-2d60-4f65-bc75-80bdeeed30d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17b127e5f60a4b5ba93243a42b6f9c29, user_id=d4ec25c162eb481b846b60a41683e7c5, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:34.830053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2338e5-2d60-4f65-bc75-80bdeeed30d3 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:34.876221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 850/1723] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:34 2022] PUT /identity/v3/projects/17b127e5f60a4b5ba93243a42b6f9c29/users/d4ec25c162eb481b846b60a41683e7c5/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:34.877993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972aaa4-e02d-4caa-8928-3e046949aaf2 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:34.910171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972aaa4-e02d-4caa-8928-3e046949aaf2 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:35.081891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10c651-3297-40dd-ad95-f675d433b327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:35.082793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10c651-3297-40dd-ad95-f675d433b327 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:35.083145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10c651-3297-40dd-ad95-f675d433b327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:35.083542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10c651-3297-40dd-ad95-f675d433b327 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:35.084291 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10c651-3297-40dd-ad95-f675d433b327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:35.085976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10c651-3297-40dd-ad95-f675d433b327 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:35.125212 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 874/1724] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:35.129271 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2338e5-2d60-4f65-bc75-80bdeeed30d3 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:35.142642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2338e5-2d60-4f65-bc75-80bdeeed30d3 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:35.148524 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9be66-80ca-4061-ae0f-ab7149d12af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:35.149733 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9be66-80ca-4061-ae0f-ab7149d12af2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:35.150020 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9be66-80ca-4061-ae0f-ab7149d12af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:35.150928 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9be66-80ca-4061-ae0f-ab7149d12af2 admin admin] PATH_INFO: `/v3/projects/17b127e5f60a4b5ba93243a42b6f9c29/users/d4ec25c162eb481b846b60a41683e7c5/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:35.154041 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d9be66-80ca-4061-ae0f-ab7149d12af2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17b127e5f60a4b5ba93243a42b6f9c29, user_id=d4ec25c162eb481b846b60a41683e7c5, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:35.157764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d9be66-80ca-4061-ae0f-ab7149d12af2 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:35.161115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:35.196452 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 851/1725] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:35 2022] PUT /identity/v3/projects/17b127e5f60a4b5ba93243a42b6f9c29/users/d4ec25c162eb481b846b60a41683e7c5/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:35.198921 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684fa7f-477f-4485-9f4b-4589f67e38df None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:35.199209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684fa7f-477f-4485-9f4b-4589f67e38df None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:35.199506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684fa7f-477f-4485-9f4b-4589f67e38df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:35.279807 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-4684fa7f-477f-4485-9f4b-4589f67e38df None None] MFA Rules not processed for user `d4ec25c162eb481b846b60a41683e7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:35.287476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4684fa7f-477f-4485-9f4b-4589f67e38df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:35.542673 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 875/1726] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:35.552976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9be66-80ca-4061-ae0f-ab7149d12af2 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:35.554747 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-74d9be66-80ca-4061-ae0f-ab7149d12af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:35.584761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b7f8e-b041-413b-ab11-e525531d46dd None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:35.585268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b7f8e-b041-413b-ab11-e525531d46dd None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:35.585823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b7f8e-b041-413b-ab11-e525531d46dd None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:35.588768 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 876/1727] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 23 07:12:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:12:35.642077 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cffdc6-f155-405d-a6c9-78b813742ac2 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:35.642999 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cffdc6-f155-405d-a6c9-78b813742ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:35.643647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cffdc6-f155-405d-a6c9-78b813742ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:35.716706 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-c4cffdc6-f155-405d-a6c9-78b813742ac2 None None] MFA Rules not processed for user `a9b0f54c988e43bda474f978a69fa4fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:35.721235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c4cffdc6-f155-405d-a6c9-78b813742ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:35.771858 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9be66-80ca-4061-ae0f-ab7149d12af2 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:35.789674 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a18996-fe00-443b-bc41-8ce1a8699141 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:35.795565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a18996-fe00-443b-bc41-8ce1a8699141 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:35.796445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a18996-fe00-443b-bc41-8ce1a8699141 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:35.797904 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a18996-fe00-443b-bc41-8ce1a8699141 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:35.821079 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a18996-fe00-443b-bc41-8ce1a8699141 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:35.826186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a18996-fe00-443b-bc41-8ce1a8699141 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:35.850451 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 852/1728] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:35.851939 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a18996-fe00-443b-bc41-8ce1a8699141 service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:35.854650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-34a18996-fe00-443b-bc41-8ce1a8699141 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:35.908726 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:35.963825 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 877/1729] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 23 07:12:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:35.995216 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cffdc6-f155-405d-a6c9-78b813742ac2 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:36.026203 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cffdc6-f155-405d-a6c9-78b813742ac2 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:36.043940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626ccbe-4058-42f1-a6fe-a9420308d925 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9b0f54c988e43bda474f978a69fa4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:36.045085 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626ccbe-4058-42f1-a6fe-a9420308d925 service placement] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:36.045510 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626ccbe-4058-42f1-a6fe-a9420308d925 service placement] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:36.045935 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626ccbe-4058-42f1-a6fe-a9420308d925 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:36.078685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a18996-fe00-443b-bc41-8ce1a8699141 service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:36.085716 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9626ccbe-4058-42f1-a6fe-a9420308d925 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:36.087482 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9626ccbe-4058-42f1-a6fe-a9420308d925 service placement] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:36.093728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573bde64-8f42-4270-8d87-492ba29108b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:36.094572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bde64-8f42-4270-8d87-492ba29108b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:36.094995 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bde64-8f42-4270-8d87-492ba29108b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:36.098400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bde64-8f42-4270-8d87-492ba29108b0 admin admin] PATH_INFO: `/v3/users/9cda314fd3604be6b37669a2524c0e3c` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:36.105983 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573bde64-8f42-4270-8d87-492ba29108b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cda314fd3604be6b37669a2524c0e3c)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:36.112500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573bde64-8f42-4270-8d87-492ba29108b0 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:36.143573 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 878/1730] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:36.212302 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-573bde64-8f42-4270-8d87-492ba29108b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cda314fd3604be6b37669a2524c0e3c'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:36.213575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-573bde64-8f42-4270-8d87-492ba29108b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cda314fd3604be6b37669a2524c0e3c'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:36.214444 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-573bde64-8f42-4270-8d87-492ba29108b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cda314fd3604be6b37669a2524c0e3c'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:36.225391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626ccbe-4058-42f1-a6fe-a9420308d925 service placement] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:36.227263 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9626ccbe-4058-42f1-a6fe-a9420308d925 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:36.234782 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-573bde64-8f42-4270-8d87-492ba29108b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cda314fd3604be6b37669a2524c0e3c'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:36.274011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626ccbe-4058-42f1-a6fe-a9420308d925 service placement] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:36.288528 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-573bde64-8f42-4270-8d87-492ba29108b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cda314fd3604be6b37669a2524c0e3c'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:36.292897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335dcb4-f5fb-4861-8700-65df49195993 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:36.293099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335dcb4-f5fb-4861-8700-65df49195993 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:36.293381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335dcb4-f5fb-4861-8700-65df49195993 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:36.293636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335dcb4-f5fb-4861-8700-65df49195993 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:36.314529 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4335dcb4-f5fb-4861-8700-65df49195993 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:36.322649 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 853/1731] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:12:35 2022] DELETE /identity/v3/users/9cda314fd3604be6b37669a2524c0e3c => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:36.558919 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335dcb4-f5fb-4861-8700-65df49195993 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:36.562313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335dcb4-f5fb-4861-8700-65df49195993 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:36.620571 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 879/1732] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:12:36.877638 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf6602-4d39-441d-ba7a-72b1e6f4b74b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:36.878369 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf6602-4d39-441d-ba7a-72b1e6f4b74b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:36.878812 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf6602-4d39-441d-ba7a-72b1e6f4b74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:36.934721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-0aaf6602-4d39-441d-ba7a-72b1e6f4b74b None None] MFA Rules not processed for user `471dc5c1db64411ab1d80ad6c038e519`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:36.937941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-0aaf6602-4d39-441d-ba7a-72b1e6f4b74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:36.967300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:37.089881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 854/1733] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:37.658675 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8e7e7-9eff-42b9-9b55-5ba8ec39ffff None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:37.659964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8e7e7-9eff-42b9-9b55-5ba8ec39ffff None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:37.660600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8e7e7-9eff-42b9-9b55-5ba8ec39ffff None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:37.664025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 880/1734] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 23 07:12:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:12:37.785312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c409dc4-cd9f-4cbc-b703-098966b85dcf None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:37.788176 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c409dc4-cd9f-4cbc-b703-098966b85dcf None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:37.788607 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c409dc4-cd9f-4cbc-b703-098966b85dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:37.843261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8e7e7-9eff-42b9-9b55-5ba8ec39ffff None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:37.845521 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-f1a8e7e7-9eff-42b9-9b55-5ba8ec39ffff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:37.882661 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-6c409dc4-cd9f-4cbc-b703-098966b85dcf None None] MFA Rules not processed for user `a8d2cd93042c4583b63afe629d47fce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:37.891844 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-6c409dc4-cd9f-4cbc-b703-098966b85dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:38.142837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8e7e7-9eff-42b9-9b55-5ba8ec39ffff None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:38.173182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59b8a3-46dc-479e-86bf-cb8aacddbeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:38.177874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59b8a3-46dc-479e-86bf-cb8aacddbeb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:38.180805 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59b8a3-46dc-479e-86bf-cb8aacddbeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:38.183067 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59b8a3-46dc-479e-86bf-cb8aacddbeb6 admin admin] PATH_INFO: `/v3/projects/c85912dcdc424208a709c4cc4959ae63` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:38.206915 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59b8a3-46dc-479e-86bf-cb8aacddbeb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c85912dcdc424208a709c4cc4959ae63)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:38.216047 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59b8a3-46dc-479e-86bf-cb8aacddbeb6 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:38.221192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 855/1735] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 23 07:12:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:38.361780 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-bd59b8a3-46dc-479e-86bf-cb8aacddbeb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c85912dcdc424208a709c4cc4959ae63', 'request_id': 'req-bd59b8a3-46dc-479e-86bf-cb8aacddbeb6'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:38.380417 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-bd59b8a3-46dc-479e-86bf-cb8aacddbeb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c85912dcdc424208a709c4cc4959ae63', 'request_id': 'req-bd59b8a3-46dc-479e-86bf-cb8aacddbeb6'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:38.395274 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-bd59b8a3-46dc-479e-86bf-cb8aacddbeb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c85912dcdc424208a709c4cc4959ae63', 'request_id': 'req-bd59b8a3-46dc-479e-86bf-cb8aacddbeb6'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:38.422590 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-bd59b8a3-46dc-479e-86bf-cb8aacddbeb6 admin admin] The token cache is being invalidate because project c85912dcdc424208a709c4cc4959ae63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:12:38.422892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-bd59b8a3-46dc-479e-86bf-cb8aacddbeb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:38.424855 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-bd59b8a3-46dc-479e-86bf-cb8aacddbeb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:38.431805 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 881/1736] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:12:37 2022] DELETE /identity/v3/projects/c85912dcdc424208a709c4cc4959ae63 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:38.436267 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c409dc4-cd9f-4cbc-b703-098966b85dcf None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:38.451521 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c409dc4-cd9f-4cbc-b703-098966b85dcf None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:38.703023 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607294c-0bba-4352-8ac8-1c582932fd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:38.704005 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607294c-0bba-4352-8ac8-1c582932fd65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:38.705018 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607294c-0bba-4352-8ac8-1c582932fd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:38.705018 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607294c-0bba-4352-8ac8-1c582932fd65 admin admin] PATH_INFO: `/v3/users/6655cece7a214bc0b11d61015444e6a5` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:38.712771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607294c-0bba-4352-8ac8-1c582932fd65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6655cece7a214bc0b11d61015444e6a5)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:38.715080 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607294c-0bba-4352-8ac8-1c582932fd65 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:38.873055 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-3607294c-0bba-4352-8ac8-1c582932fd65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6655cece7a214bc0b11d61015444e6a5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:38.877919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-3607294c-0bba-4352-8ac8-1c582932fd65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6655cece7a214bc0b11d61015444e6a5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:38.878683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-3607294c-0bba-4352-8ac8-1c582932fd65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6655cece7a214bc0b11d61015444e6a5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:38.900090 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:38.903337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-3607294c-0bba-4352-8ac8-1c582932fd65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6655cece7a214bc0b11d61015444e6a5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:38.924584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-3607294c-0bba-4352-8ac8-1c582932fd65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6655cece7a214bc0b11d61015444e6a5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:38.964389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 856/1737] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:12:38 2022] DELETE /identity/v3/users/6655cece7a214bc0b11d61015444e6a5 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:39.326048 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59b8a3-46dc-479e-86bf-cb8aacddbeb6 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:39.327566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-bd59b8a3-46dc-479e-86bf-cb8aacddbeb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:39.338525 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:39.361655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59b8a3-46dc-479e-86bf-cb8aacddbeb6 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:39.526858 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607294c-0bba-4352-8ac8-1c582932fd65 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:39.528546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3607294c-0bba-4352-8ac8-1c582932fd65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:39.534351 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9ad76-1a59-4207-8a6f-268e4e7d734d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:39.535322 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9ad76-1a59-4207-8a6f-268e4e7d734d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:39.535863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9ad76-1a59-4207-8a6f-268e4e7d734d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:39.536200 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9ad76-1a59-4207-8a6f-268e4e7d734d admin admin] PATH_INFO: `/v3/projects/7aeba7cc278d437b91761334473f2ac2` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:39.537860 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9ad76-1a59-4207-8a6f-268e4e7d734d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aeba7cc278d437b91761334473f2ac2)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:39.540517 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9ad76-1a59-4207-8a6f-268e4e7d734d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:39.639256 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5ac9ad76-1a59-4207-8a6f-268e4e7d734d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aeba7cc278d437b91761334473f2ac2', 'request_id': 'req-5ac9ad76-1a59-4207-8a6f-268e4e7d734d'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:39.668223 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5ac9ad76-1a59-4207-8a6f-268e4e7d734d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aeba7cc278d437b91761334473f2ac2', 'request_id': 'req-5ac9ad76-1a59-4207-8a6f-268e4e7d734d'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:39.696075 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5ac9ad76-1a59-4207-8a6f-268e4e7d734d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aeba7cc278d437b91761334473f2ac2', 'request_id': 'req-5ac9ad76-1a59-4207-8a6f-268e4e7d734d'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:39.738683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5ac9ad76-1a59-4207-8a6f-268e4e7d734d admin admin] The token cache is being invalidate because project 7aeba7cc278d437b91761334473f2ac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:12:39.738980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5ac9ad76-1a59-4207-8a6f-268e4e7d734d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:39.741504 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5ac9ad76-1a59-4207-8a6f-268e4e7d734d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:39.748863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 882/1738] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:12:39 2022] DELETE /identity/v3/projects/7aeba7cc278d437b91761334473f2ac2 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:39.753069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d6e5e-7b07-4f24-835c-615231ffde0b None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:39.753403 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d6e5e-7b07-4f24-835c-615231ffde0b None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:39.753635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d6e5e-7b07-4f24-835c-615231ffde0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:39.758513 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607294c-0bba-4352-8ac8-1c582932fd65 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:39.840945 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-189d6e5e-7b07-4f24-835c-615231ffde0b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:39.842100 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-189d6e5e-7b07-4f24-835c-615231ffde0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:39.966123 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712a2e9-3e23-40ad-886e-f7ba8fdaf6f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:39.968659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712a2e9-3e23-40ad-886e-f7ba8fdaf6f3 service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:39.969374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712a2e9-3e23-40ad-886e-f7ba8fdaf6f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:39.970089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712a2e9-3e23-40ad-886e-f7ba8fdaf6f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:39.984426 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2712a2e9-3e23-40ad-886e-f7ba8fdaf6f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:40.017126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 883/1739] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:40.026802 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea1dab-6219-42eb-be6f-dd6afd6ed4a7 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:40.027003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea1dab-6219-42eb-be6f-dd6afd6ed4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:40.027289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea1dab-6219-42eb-be6f-dd6afd6ed4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:40.032528 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712a2e9-3e23-40ad-886e-f7ba8fdaf6f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:40.034776 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712a2e9-3e23-40ad-886e-f7ba8fdaf6f3 service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:40.063961 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 857/1740] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:12:40.108873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-c4ea1dab-6219-42eb-be6f-dd6afd6ed4a7 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:40.110001 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c4ea1dab-6219-42eb-be6f-dd6afd6ed4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:40.116038 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff89c3f-a3fe-43f0-b3dc-21d2ca6f6b00 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:40.116719 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff89c3f-a3fe-43f0-b3dc-21d2ca6f6b00 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:40.117205 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff89c3f-a3fe-43f0-b3dc-21d2ca6f6b00 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:40.120004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 858/1741] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 23 07:12:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:12:40.130768 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 884/1742] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:12:40.136945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff89c3f-a3fe-43f0-b3dc-21d2ca6f6b00 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:40.145259 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed70f4-b191-4a35-8cd3-98bba19ba371 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:40.145558 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed70f4-b191-4a35-8cd3-98bba19ba371 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:40.145839 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed70f4-b191-4a35-8cd3-98bba19ba371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:40.200385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-3fed70f4-b191-4a35-8cd3-98bba19ba371 None None] MFA Rules not processed for user `a8d2cd93042c4583b63afe629d47fce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:40.201629 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3fed70f4-b191-4a35-8cd3-98bba19ba371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:40.207398 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff89c3f-a3fe-43f0-b3dc-21d2ca6f6b00 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:40.220593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 885/1743] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 23 07:12:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:40.222431 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdbb396-38db-4b56-a64b-937cb89bc1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:40.224219 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbb396-38db-4b56-a64b-937cb89bc1fe admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:40.227616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbb396-38db-4b56-a64b-937cb89bc1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:40.228719 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbb396-38db-4b56-a64b-937cb89bc1fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:40.231231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdbb396-38db-4b56-a64b-937cb89bc1fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:12:40.232278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdbb396-38db-4b56-a64b-937cb89bc1fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:40.235115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdbb396-38db-4b56-a64b-937cb89bc1fe admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:40.276005 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 859/1744] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:12:40.280011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed70f4-b191-4a35-8cd3-98bba19ba371 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:40.292149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed70f4-b191-4a35-8cd3-98bba19ba371 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:40.308970 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47604c58-4608-4833-b007-3ac09920a5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:40.309712 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47604c58-4608-4833-b007-3ac09920a5c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:40.310051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47604c58-4608-4833-b007-3ac09920a5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:40.310400 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47604c58-4608-4833-b007-3ac09920a5c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:40.311124 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47604c58-4608-4833-b007-3ac09920a5c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:40.313369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47604c58-4608-4833-b007-3ac09920a5c7 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:40.342629 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:40.374631 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 886/1745] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:40 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:40.379321 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdbb396-38db-4b56-a64b-937cb89bc1fe admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:40.396587 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdbb396-38db-4b56-a64b-937cb89bc1fe admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:40.572573 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:40.614814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd1d8e-988c-46d0-8cef-7b1719137d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:40.615727 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd1d8e-988c-46d0-8cef-7b1719137d8d admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:40.616149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd1d8e-988c-46d0-8cef-7b1719137d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:40.617656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd1d8e-988c-46d0-8cef-7b1719137d8d admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:40.618506 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd1d8e-988c-46d0-8cef-7b1719137d8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:40.620657 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd1d8e-988c-46d0-8cef-7b1719137d8d admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:40.675483 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 860/1746] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:40 2022] POST /identity/v3/users => generated 288 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:40.683716 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47604c58-4608-4833-b007-3ac09920a5c7 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:40.704083 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47604c58-4608-4833-b007-3ac09920a5c7 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:40.720334 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24580d71-7353-4d70-85b0-f65b2ccc1271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:40.720967 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24580d71-7353-4d70-85b0-f65b2ccc1271 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:40.721251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24580d71-7353-4d70-85b0-f65b2ccc1271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:40.721524 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24580d71-7353-4d70-85b0-f65b2ccc1271 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:40.722173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24580d71-7353-4d70-85b0-f65b2ccc1271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:40.723823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24580d71-7353-4d70-85b0-f65b2ccc1271 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:40.763880 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 887/1747] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:40.767081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd1d8e-988c-46d0-8cef-7b1719137d8d admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:40.799226 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd1d8e-988c-46d0-8cef-7b1719137d8d admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:40.821088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d0664-c196-4f53-bafa-696acba40857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:40.821088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d0664-c196-4f53-bafa-696acba40857 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:40.821088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d0664-c196-4f53-bafa-696acba40857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:40.821088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d0664-c196-4f53-bafa-696acba40857 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:40.821967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d0664-c196-4f53-bafa-696acba40857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:40.825646 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d0664-c196-4f53-bafa-696acba40857 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:40.848587 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24580d71-7353-4d70-85b0-f65b2ccc1271 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:40.850354 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-24580d71-7353-4d70-85b0-f65b2ccc1271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:40.868851 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 861/1748] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:40.870074 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d0664-c196-4f53-bafa-696acba40857 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:40.910332 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d0664-c196-4f53-bafa-696acba40857 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:40.921969 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cee57-86a1-4d7d-b59c-72e9bcfcaf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:40.923226 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cee57-86a1-4d7d-b59c-72e9bcfcaf58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:40.923924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cee57-86a1-4d7d-b59c-72e9bcfcaf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:40.924408 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cee57-86a1-4d7d-b59c-72e9bcfcaf58 admin admin] PATH_INFO: `/v3/projects/afccf25909bb4f84a61397e1bf6d87cd/users/ae739cd6305148cc9f99afe62f9079c7/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:40.989327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667cee57-86a1-4d7d-b59c-72e9bcfcaf58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afccf25909bb4f84a61397e1bf6d87cd, user_id=ae739cd6305148cc9f99afe62f9079c7, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:40.994955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667cee57-86a1-4d7d-b59c-72e9bcfcaf58 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:41.026253 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 862/1749] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:40 2022] PUT /identity/v3/projects/afccf25909bb4f84a61397e1bf6d87cd/users/ae739cd6305148cc9f99afe62f9079c7/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:41.039916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24580d71-7353-4d70-85b0-f65b2ccc1271 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:41.047236 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8800d-a2bf-42f8-8786-ecedd3816d23 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:41.048052 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8800d-a2bf-42f8-8786-ecedd3816d23 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:41.048533 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8800d-a2bf-42f8-8786-ecedd3816d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:41.102289 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-76d8800d-a2bf-42f8-8786-ecedd3816d23 None None] MFA Rules not processed for user `ae739cd6305148cc9f99afe62f9079c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:41.103681 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-76d8800d-a2bf-42f8-8786-ecedd3816d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:41.138440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85275c91-4cae-41bf-b51a-908461ceb786 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:41.142633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85275c91-4cae-41bf-b51a-908461ceb786 service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:41.143012 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85275c91-4cae-41bf-b51a-908461ceb786 service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:41.143365 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85275c91-4cae-41bf-b51a-908461ceb786 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:41.154531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85275c91-4cae-41bf-b51a-908461ceb786 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:41.156961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85275c91-4cae-41bf-b51a-908461ceb786 service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:41.197313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 888/1750] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:12:41.216580 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 863/1751] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:12:41.220786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85275c91-4cae-41bf-b51a-908461ceb786 service nova] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:41.353978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85275c91-4cae-41bf-b51a-908461ceb786 service nova] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:41.360643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db71bcf-3e73-47ba-b41a-259557f1272c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:41.362105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db71bcf-3e73-47ba-b41a-259557f1272c admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:41.362953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db71bcf-3e73-47ba-b41a-259557f1272c admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:41.363376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db71bcf-3e73-47ba-b41a-259557f1272c admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:41.364200 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db71bcf-3e73-47ba-b41a-259557f1272c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:41.366314 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db71bcf-3e73-47ba-b41a-259557f1272c admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:41.385032 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:41.403627 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 889/1752] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:41 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:12:41.405680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8800d-a2bf-42f8-8786-ecedd3816d23 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:41.419123 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8800d-a2bf-42f8-8786-ecedd3816d23 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:41.430614 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db71bcf-3e73-47ba-b41a-259557f1272c admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:41.432940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9db71bcf-3e73-47ba-b41a-259557f1272c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:41.494322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aecaaa4-5fb0-4bfa-bf34-5bacda2c7a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:41.494930 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aecaaa4-5fb0-4bfa-bf34-5bacda2c7a4d admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:41.495207 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aecaaa4-5fb0-4bfa-bf34-5bacda2c7a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:41.495506 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aecaaa4-5fb0-4bfa-bf34-5bacda2c7a4d admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:41.496274 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aecaaa4-5fb0-4bfa-bf34-5bacda2c7a4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:41.498323 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aecaaa4-5fb0-4bfa-bf34-5bacda2c7a4d admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:41.504968 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db71bcf-3e73-47ba-b41a-259557f1272c admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:41.510472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94962dc7-b5bf-4f84-aade-5431619948e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:41.511126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94962dc7-b5bf-4f84-aade-5431619948e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:41.511509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94962dc7-b5bf-4f84-aade-5431619948e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:41.511707 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94962dc7-b5bf-4f84-aade-5431619948e0 admin admin] PATH_INFO: `/v3/users/78b086e77cca4868ba85210397ff94e4` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:41.512941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94962dc7-b5bf-4f84-aade-5431619948e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b086e77cca4868ba85210397ff94e4)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:41.514941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94962dc7-b5bf-4f84-aade-5431619948e0 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:41.522798 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 864/1753] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:41 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:41.526955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aecaaa4-5fb0-4bfa-bf34-5bacda2c7a4d admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:41.537399 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aecaaa4-5fb0-4bfa-bf34-5bacda2c7a4d admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:41.543155 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ea786-8b46-44e8-8af5-7a1e7d09f55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:41.544142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ea786-8b46-44e8-8af5-7a1e7d09f55e admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:41.544485 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ea786-8b46-44e8-8af5-7a1e7d09f55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:41.544881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ea786-8b46-44e8-8af5-7a1e7d09f55e admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:41.545631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ea786-8b46-44e8-8af5-7a1e7d09f55e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:41.547453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ea786-8b46-44e8-8af5-7a1e7d09f55e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:41.559456 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 865/1754] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:12:41.563507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ea786-8b46-44e8-8af5-7a1e7d09f55e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:41.569844 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:41.574201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ea786-8b46-44e8-8af5-7a1e7d09f55e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:41.576388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-94962dc7-b5bf-4f84-aade-5431619948e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b086e77cca4868ba85210397ff94e4'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:41.577409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-94962dc7-b5bf-4f84-aade-5431619948e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b086e77cca4868ba85210397ff94e4'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:41.578158 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-94962dc7-b5bf-4f84-aade-5431619948e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b086e77cca4868ba85210397ff94e4'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:41.580304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529dbf4-b3cf-4bbd-9dd9-26e4feea99f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:41.581465 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529dbf4-b3cf-4bbd-9dd9-26e4feea99f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:41.581747 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529dbf4-b3cf-4bbd-9dd9-26e4feea99f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:41.582024 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529dbf4-b3cf-4bbd-9dd9-26e4feea99f8 admin admin] PATH_INFO: `/v3/projects/2ae0afc91d574af79906ec9db97324b7/users/dcbee6a4f7e649fabbf63b1c9c7bfa24/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:41.588096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-94962dc7-b5bf-4f84-aade-5431619948e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b086e77cca4868ba85210397ff94e4'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:41.598461 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529dbf4-b3cf-4bbd-9dd9-26e4feea99f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae0afc91d574af79906ec9db97324b7, user_id=dcbee6a4f7e649fabbf63b1c9c7bfa24, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:41.601212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529dbf4-b3cf-4bbd-9dd9-26e4feea99f8 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:41.602167 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-94962dc7-b5bf-4f84-aade-5431619948e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b086e77cca4868ba85210397ff94e4'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:41.617283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 866/1755] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:41 2022] PUT /identity/v3/projects/2ae0afc91d574af79906ec9db97324b7/users/dcbee6a4f7e649fabbf63b1c9c7bfa24/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:12:41.619025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 890/1756] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:12:41 2022] DELETE /identity/v3/users/78b086e77cca4868ba85210397ff94e4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:41.628856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529dbf4-b3cf-4bbd-9dd9-26e4feea99f8 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:41.628856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c529dbf4-b3cf-4bbd-9dd9-26e4feea99f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:41.629041 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aebb48-560e-4ee4-b75d-233d9e53635c None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:41.629041 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aebb48-560e-4ee4-b75d-233d9e53635c None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:41.629041 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aebb48-560e-4ee4-b75d-233d9e53635c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:41.644870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-60aebb48-560e-4ee4-b75d-233d9e53635c None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:41.646595 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-60aebb48-560e-4ee4-b75d-233d9e53635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:41.703445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529dbf4-b3cf-4bbd-9dd9-26e4feea99f8 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:41.722527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 891/1757] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:12:41.732753 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aebb48-560e-4ee4-b75d-233d9e53635c None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:41.734241 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-60aebb48-560e-4ee4-b75d-233d9e53635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:41.751779 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:41.767219 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454078cd-bfc3-4b0b-be73-36b17bca35d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:41.767876 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454078cd-bfc3-4b0b-be73-36b17bca35d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:41.768158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454078cd-bfc3-4b0b-be73-36b17bca35d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:41.768430 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454078cd-bfc3-4b0b-be73-36b17bca35d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:41.769081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454078cd-bfc3-4b0b-be73-36b17bca35d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:41.770744 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454078cd-bfc3-4b0b-be73-36b17bca35d2 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:41.780863 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 867/1758] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:12:41.785266 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454078cd-bfc3-4b0b-be73-36b17bca35d2 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:41.795263 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454078cd-bfc3-4b0b-be73-36b17bca35d2 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:41.796134 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aebb48-560e-4ee4-b75d-233d9e53635c None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:41.799841 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f27c2d-c481-455f-a35a-a40d2a507459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:41.800038 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4e2c5-c136-421d-8dec-45fa2d651403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:41.800442 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f27c2d-c481-455f-a35a-a40d2a507459 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:41.800688 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4e2c5-c136-421d-8dec-45fa2d651403 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:41.800784 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f27c2d-c481-455f-a35a-a40d2a507459 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:41.800970 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4e2c5-c136-421d-8dec-45fa2d651403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:41.801065 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f27c2d-c481-455f-a35a-a40d2a507459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:41.801247 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4e2c5-c136-421d-8dec-45fa2d651403 admin admin] PATH_INFO: `/v3/projects/2ae0afc91d574af79906ec9db97324b7/users/dcbee6a4f7e649fabbf63b1c9c7bfa24/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:41.803173 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4e2c5-c136-421d-8dec-45fa2d651403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae0afc91d574af79906ec9db97324b7, user_id=dcbee6a4f7e649fabbf63b1c9c7bfa24, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:41.805910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4e2c5-c136-421d-8dec-45fa2d651403 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:41.810055 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f27c2d-c481-455f-a35a-a40d2a507459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:41.811506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f27c2d-c481-455f-a35a-a40d2a507459 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:41.816899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 868/1759] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:41 2022] PUT /identity/v3/projects/2ae0afc91d574af79906ec9db97324b7/users/dcbee6a4f7e649fabbf63b1c9c7bfa24/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:12:41.825277 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 892/1760] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:12:41.828253 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1d213-af4f-4731-acb4-96a4ce3235c0 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:41.828539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1d213-af4f-4731-acb4-96a4ce3235c0 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:41.828809 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1d213-af4f-4731-acb4-96a4ce3235c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:41.857536 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-b2a1d213-af4f-4731-acb4-96a4ce3235c0 None None] MFA Rules not processed for user `dcbee6a4f7e649fabbf63b1c9c7bfa24`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:41.858752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b2a1d213-af4f-4731-acb4-96a4ce3235c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:41.922343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:41.935634 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 869/1761] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:12:41.948574 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f27c2d-c481-455f-a35a-a40d2a507459 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:41.950038 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b3f27c2d-c481-455f-a35a-a40d2a507459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:42.016644 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f27c2d-c481-455f-a35a-a40d2a507459 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:42.066705 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f41f38-cd35-401b-b60e-c8283c943c8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:42.067324 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f41f38-cd35-401b-b60e-c8283c943c8c service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:42.067651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f41f38-cd35-401b-b60e-c8283c943c8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:42.067947 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f41f38-cd35-401b-b60e-c8283c943c8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:42.069764 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-49f41f38-cd35-401b-b60e-c8283c943c8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:42.104459 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1d213-af4f-4731-acb4-96a4ce3235c0 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:42.106661 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b2a1d213-af4f-4731-acb4-96a4ce3235c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:42.144167 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f41f38-cd35-401b-b60e-c8283c943c8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:42.145824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f41f38-cd35-401b-b60e-c8283c943c8c service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:42.156592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 893/1762] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:42.195311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1d213-af4f-4731-acb4-96a4ce3235c0 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:42.200204 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14030fe-b620-444a-baa3-a5766721e966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:42.200873 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14030fe-b620-444a-baa3-a5766721e966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:42.201148 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14030fe-b620-444a-baa3-a5766721e966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:42.201564 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14030fe-b620-444a-baa3-a5766721e966 admin admin] PATH_INFO: `/v3/projects/72debe469f5744d698f5967399563c7a` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:42.202692 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14030fe-b620-444a-baa3-a5766721e966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72debe469f5744d698f5967399563c7a)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:42.205223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14030fe-b620-444a-baa3-a5766721e966 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:42.262652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e14030fe-b620-444a-baa3-a5766721e966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72debe469f5744d698f5967399563c7a', 'request_id': 'req-e14030fe-b620-444a-baa3-a5766721e966'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:42.270952 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e14030fe-b620-444a-baa3-a5766721e966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72debe469f5744d698f5967399563c7a', 'request_id': 'req-e14030fe-b620-444a-baa3-a5766721e966'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:42.281147 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e14030fe-b620-444a-baa3-a5766721e966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72debe469f5744d698f5967399563c7a', 'request_id': 'req-e14030fe-b620-444a-baa3-a5766721e966'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:42.294204 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e14030fe-b620-444a-baa3-a5766721e966 admin admin] The token cache is being invalidate because project 72debe469f5744d698f5967399563c7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:12:42.294788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e14030fe-b620-444a-baa3-a5766721e966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:42.295374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e14030fe-b620-444a-baa3-a5766721e966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:42.297952 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 870/1763] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:12:42 2022] DELETE /identity/v3/projects/72debe469f5744d698f5967399563c7a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:42.302295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f41f38-cd35-401b-b60e-c8283c943c8c service cinder] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:42.317982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f41f38-cd35-401b-b60e-c8283c943c8c service cinder] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:42.346663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:42.384258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96121e-e121-4236-8663-890a7b573e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:42.385091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96121e-e121-4236-8663-890a7b573e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:42.385451 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96121e-e121-4236-8663-890a7b573e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:42.385726 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96121e-e121-4236-8663-890a7b573e80 admin admin] PATH_INFO: `/v3/users/61208b1bbc02419a8aa708270964b2a4` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:42.386904 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96121e-e121-4236-8663-890a7b573e80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61208b1bbc02419a8aa708270964b2a4)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:42.388901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96121e-e121-4236-8663-890a7b573e80 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:42.465088 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-be96121e-e121-4236-8663-890a7b573e80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61208b1bbc02419a8aa708270964b2a4'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:42.465961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-be96121e-e121-4236-8663-890a7b573e80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61208b1bbc02419a8aa708270964b2a4'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:42.466571 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-be96121e-e121-4236-8663-890a7b573e80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61208b1bbc02419a8aa708270964b2a4'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:42.476626 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-be96121e-e121-4236-8663-890a7b573e80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61208b1bbc02419a8aa708270964b2a4'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:42.490617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-be96121e-e121-4236-8663-890a7b573e80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61208b1bbc02419a8aa708270964b2a4'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:42.509581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 894/1764] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:12:42 2022] DELETE /identity/v3/users/61208b1bbc02419a8aa708270964b2a4 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:12:42.811712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14030fe-b620-444a-baa3-a5766721e966 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:42.813300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e14030fe-b620-444a-baa3-a5766721e966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:42.829208 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14030fe-b620-444a-baa3-a5766721e966 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:42.891844 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17861da6-069e-4e31-812f-96990af04dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:42.892510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17861da6-069e-4e31-812f-96990af04dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:42.892773 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17861da6-069e-4e31-812f-96990af04dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:42.893053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17861da6-069e-4e31-812f-96990af04dc8 admin admin] PATH_INFO: `/v3/projects/cbfe4f25844d4ae593c4bcc8bc55b66e` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:42.894265 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17861da6-069e-4e31-812f-96990af04dc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbfe4f25844d4ae593c4bcc8bc55b66e)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:42.896263 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17861da6-069e-4e31-812f-96990af04dc8 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:42.931266 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:42.948263 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-17861da6-069e-4e31-812f-96990af04dc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbfe4f25844d4ae593c4bcc8bc55b66e', 'request_id': 'req-17861da6-069e-4e31-812f-96990af04dc8'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:42.953748 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-17861da6-069e-4e31-812f-96990af04dc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbfe4f25844d4ae593c4bcc8bc55b66e', 'request_id': 'req-17861da6-069e-4e31-812f-96990af04dc8'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:42.963097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-17861da6-069e-4e31-812f-96990af04dc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbfe4f25844d4ae593c4bcc8bc55b66e', 'request_id': 'req-17861da6-069e-4e31-812f-96990af04dc8'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:42.974374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-17861da6-069e-4e31-812f-96990af04dc8 admin admin] The token cache is being invalidate because project cbfe4f25844d4ae593c4bcc8bc55b66e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:12:42.974665 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-17861da6-069e-4e31-812f-96990af04dc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:42.975542 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-17861da6-069e-4e31-812f-96990af04dc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:42.977013 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 871/1765] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:12:42 2022] DELETE /identity/v3/projects/cbfe4f25844d4ae593c4bcc8bc55b66e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:42.982001 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1a193-09d7-46c9-af0d-7c529e4beb5b None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:42.982288 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1a193-09d7-46c9-af0d-7c529e4beb5b None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:42.982580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1a193-09d7-46c9-af0d-7c529e4beb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:43.004197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-4db1a193-09d7-46c9-af0d-7c529e4beb5b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:43.005428 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4db1a193-09d7-46c9-af0d-7c529e4beb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:43.067015 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 895/1766] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:43.076518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda1263-1346-4f56-ab21-6011070cf344 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:43.076731 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda1263-1346-4f56-ab21-6011070cf344 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:43.077165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda1263-1346-4f56-ab21-6011070cf344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:43.100131 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-3bda1263-1346-4f56-ab21-6011070cf344 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:43.101337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3bda1263-1346-4f56-ab21-6011070cf344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:43.111193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 872/1767] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:43.115796 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1a193-09d7-46c9-af0d-7c529e4beb5b None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:43.127193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1a193-09d7-46c9-af0d-7c529e4beb5b None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:43.133022 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08a816-8f6d-459f-b29a-802f0181a13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:43.133663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08a816-8f6d-459f-b29a-802f0181a13a admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:43.133945 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08a816-8f6d-459f-b29a-802f0181a13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:43.134279 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08a816-8f6d-459f-b29a-802f0181a13a admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:43.135122 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08a816-8f6d-459f-b29a-802f0181a13a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:12:43.135506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08a816-8f6d-459f-b29a-802f0181a13a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:43.137095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08a816-8f6d-459f-b29a-802f0181a13a admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:43.150842 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 896/1768] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:12:43.157071 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda1263-1346-4f56-ab21-6011070cf344 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:43.166877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda1263-1346-4f56-ab21-6011070cf344 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:43.172883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1560743-69d7-44b4-9d74-2d168f88ff99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:43.173501 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1560743-69d7-44b4-9d74-2d168f88ff99 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:43.173761 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1560743-69d7-44b4-9d74-2d168f88ff99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:43.174021 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1560743-69d7-44b4-9d74-2d168f88ff99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:43.174638 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1560743-69d7-44b4-9d74-2d168f88ff99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:43.176640 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1560743-69d7-44b4-9d74-2d168f88ff99 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:43.219848 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 873/1769] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:43 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:43.222310 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08a816-8f6d-459f-b29a-802f0181a13a admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:43.244675 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08a816-8f6d-459f-b29a-802f0181a13a admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:43.323213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:43.370590 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a62536b-a9d8-4c85-8905-ac069e1de034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:43.371267 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a62536b-a9d8-4c85-8905-ac069e1de034 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:43.371605 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a62536b-a9d8-4c85-8905-ac069e1de034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:43.371888 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a62536b-a9d8-4c85-8905-ac069e1de034 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:43.374734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a62536b-a9d8-4c85-8905-ac069e1de034 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:43.377629 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a62536b-a9d8-4c85-8905-ac069e1de034 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:43.410685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 897/1770] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:43 2022] POST /identity/v3/users => generated 283 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:43.414816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1560743-69d7-44b4-9d74-2d168f88ff99 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:43.425654 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1560743-69d7-44b4-9d74-2d168f88ff99 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:43.431343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d87a8-741f-49cd-88ea-41c1d3bfed16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:43.432074 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d87a8-741f-49cd-88ea-41c1d3bfed16 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:43.432491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d87a8-741f-49cd-88ea-41c1d3bfed16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:43.432774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d87a8-741f-49cd-88ea-41c1d3bfed16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:43.433459 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d87a8-741f-49cd-88ea-41c1d3bfed16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:43.435125 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d87a8-741f-49cd-88ea-41c1d3bfed16 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:43.447117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 874/1771] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:43.451150 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a62536b-a9d8-4c85-8905-ac069e1de034 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:43.468479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a62536b-a9d8-4c85-8905-ac069e1de034 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:43.474129 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e9893-fa37-43b3-a2b2-925264dcb71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:43.475007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e9893-fa37-43b3-a2b2-925264dcb71e admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:43.475250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e9893-fa37-43b3-a2b2-925264dcb71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:43.475585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e9893-fa37-43b3-a2b2-925264dcb71e admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:43.476302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e9893-fa37-43b3-a2b2-925264dcb71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:43.478251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e9893-fa37-43b3-a2b2-925264dcb71e admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:43.507524 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 898/1772] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:12:43.509574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d87a8-741f-49cd-88ea-41c1d3bfed16 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:43.527232 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d87a8-741f-49cd-88ea-41c1d3bfed16 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:43.532707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6742e82-6225-46a3-abfd-cef69529abcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:43.533539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6742e82-6225-46a3-abfd-cef69529abcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:43.534195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6742e82-6225-46a3-abfd-cef69529abcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:43.534195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6742e82-6225-46a3-abfd-cef69529abcf admin admin] PATH_INFO: `/v3/projects/802af76392bd4ad2b20866c0f6197ca2/users/47d206b049a04052a2e40efcb071ef45/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:43.559674 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6742e82-6225-46a3-abfd-cef69529abcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=802af76392bd4ad2b20866c0f6197ca2, user_id=47d206b049a04052a2e40efcb071ef45, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:43.569149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6742e82-6225-46a3-abfd-cef69529abcf admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:43.610106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d3890-cd60-4d7c-8262-0a29278f233a None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:43.610379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d3890-cd60-4d7c-8262-0a29278f233a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:43.610664 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d3890-cd60-4d7c-8262-0a29278f233a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:43.614154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 875/1773] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:43 2022] PUT /identity/v3/projects/802af76392bd4ad2b20866c0f6197ca2/users/47d206b049a04052a2e40efcb071ef45/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:43.695029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-d87d3890-cd60-4d7c-8262-0a29278f233a None None] MFA Rules not processed for user `47d206b049a04052a2e40efcb071ef45`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:43.696298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d87d3890-cd60-4d7c-8262-0a29278f233a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:43.851832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 899/1774] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:43.855697 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6742e82-6225-46a3-abfd-cef69529abcf admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:44.000387 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6742e82-6225-46a3-abfd-cef69529abcf admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:44.014975 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca54ca-8d66-4f1b-899c-1af7fcf6f179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:44.017596 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca54ca-8d66-4f1b-899c-1af7fcf6f179 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:44.018069 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca54ca-8d66-4f1b-899c-1af7fcf6f179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:44.019283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca54ca-8d66-4f1b-899c-1af7fcf6f179 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:44.020944 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca54ca-8d66-4f1b-899c-1af7fcf6f179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:44.025159 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca54ca-8d66-4f1b-899c-1af7fcf6f179 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:44.115270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 876/1775] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:43 2022] POST /identity/v3/projects => generated 365 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:12:44.121599 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d3890-cd60-4d7c-8262-0a29278f233a None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:44.152786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d3890-cd60-4d7c-8262-0a29278f233a None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:44.282149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd5c33-0688-4e1c-9d47-d2292312ddc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:44.283112 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd5c33-0688-4e1c-9d47-d2292312ddc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:44.283584 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd5c33-0688-4e1c-9d47-d2292312ddc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:44.284897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd5c33-0688-4e1c-9d47-d2292312ddc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:44.285775 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd5c33-0688-4e1c-9d47-d2292312ddc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:44.296765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd5c33-0688-4e1c-9d47-d2292312ddc1 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:44.325382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 900/1776] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:44 2022] POST /identity/v3/users => generated 289 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:12:44.329460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca54ca-8d66-4f1b-899c-1af7fcf6f179 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:44.341442 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca54ca-8d66-4f1b-899c-1af7fcf6f179 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:44.346287 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212d524-764e-4a0e-b718-aad05426fbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:44.346922 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212d524-764e-4a0e-b718-aad05426fbb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:44.347224 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212d524-764e-4a0e-b718-aad05426fbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:44.347575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212d524-764e-4a0e-b718-aad05426fbb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:44.348340 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212d524-764e-4a0e-b718-aad05426fbb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:44.350321 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212d524-764e-4a0e-b718-aad05426fbb2 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:44.365816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 877/1777] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:44.373968 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd5c33-0688-4e1c-9d47-d2292312ddc1 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:44.385203 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd5c33-0688-4e1c-9d47-d2292312ddc1 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:44.389949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1536fd-0e3b-4271-826e-848d7943105f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:44.390596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1536fd-0e3b-4271-826e-848d7943105f admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:44.390897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1536fd-0e3b-4271-826e-848d7943105f admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:44.391058 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1536fd-0e3b-4271-826e-848d7943105f admin admin] PATH_INFO: `/v3/projects/ea388c62925345fba53df92ee1201c69/users/7798edef75a046b784e6782291717843/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:44.406987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1536fd-0e3b-4271-826e-848d7943105f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea388c62925345fba53df92ee1201c69, user_id=7798edef75a046b784e6782291717843, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:44.409696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1536fd-0e3b-4271-826e-848d7943105f admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:44.425766 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 901/1778] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:44 2022] PUT /identity/v3/projects/ea388c62925345fba53df92ee1201c69/users/7798edef75a046b784e6782291717843/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:44.429918 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212d524-764e-4a0e-b718-aad05426fbb2 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:44.440372 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212d524-764e-4a0e-b718-aad05426fbb2 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:44.466258 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:44.499959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4a826-0ab3-4c49-b8e8-b89df11da2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:44.500584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4a826-0ab3-4c49-b8e8-b89df11da2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:44.500868 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4a826-0ab3-4c49-b8e8-b89df11da2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:44.501143 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4a826-0ab3-4c49-b8e8-b89df11da2e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:44.501829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4a826-0ab3-4c49-b8e8-b89df11da2e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:44.503434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4a826-0ab3-4c49-b8e8-b89df11da2e9 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:44.514794 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 878/1779] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:44.519041 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1536fd-0e3b-4271-826e-848d7943105f admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:44.529647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1536fd-0e3b-4271-826e-848d7943105f admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:44.534458 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d4e51-5a28-4d57-8206-83777694e862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:44.535116 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d4e51-5a28-4d57-8206-83777694e862 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:44.535393 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d4e51-5a28-4d57-8206-83777694e862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:44.535703 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d4e51-5a28-4d57-8206-83777694e862 admin admin] PATH_INFO: `/v3/projects/ea388c62925345fba53df92ee1201c69/users/7798edef75a046b784e6782291717843/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:44.537614 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d4e51-5a28-4d57-8206-83777694e862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea388c62925345fba53df92ee1201c69, user_id=7798edef75a046b784e6782291717843, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:44.540528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d4e51-5a28-4d57-8206-83777694e862 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:44.543599 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:44.552945 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 902/1780] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:44 2022] PUT /identity/v3/projects/ea388c62925345fba53df92ee1201c69/users/7798edef75a046b784e6782291717843/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:44.564781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a8e00-575d-45bc-a7c8-d3b88c407316 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:44.565068 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a8e00-575d-45bc-a7c8-d3b88c407316 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:44.565344 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a8e00-575d-45bc-a7c8-d3b88c407316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:44.593446 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-598a8e00-575d-45bc-a7c8-d3b88c407316 None None] MFA Rules not processed for user `7798edef75a046b784e6782291717843`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:44.594650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-598a8e00-575d-45bc-a7c8-d3b88c407316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:44.669562 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 879/1781] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:44.682141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d4e51-5a28-4d57-8206-83777694e862 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:44.684365 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-be7d4e51-5a28-4d57-8206-83777694e862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:44.749309 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d4e51-5a28-4d57-8206-83777694e862 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:44.753295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb03125-8d9f-4926-8883-560816d3cc2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:44.753992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb03125-8d9f-4926-8883-560816d3cc2a service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:44.754272 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb03125-8d9f-4926-8883-560816d3cc2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:44.754541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb03125-8d9f-4926-8883-560816d3cc2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:44.764081 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb03125-8d9f-4926-8883-560816d3cc2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:44.765532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb03125-8d9f-4926-8883-560816d3cc2a service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:44.775230 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 903/1782] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:44.854914 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a8e00-575d-45bc-a7c8-d3b88c407316 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:44.865018 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a8e00-575d-45bc-a7c8-d3b88c407316 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:44.868833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccd140-9fff-4ad7-8225-03f1ef7f48f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:44.869434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccd140-9fff-4ad7-8225-03f1ef7f48f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:44.869706 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccd140-9fff-4ad7-8225-03f1ef7f48f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:44.869980 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccd140-9fff-4ad7-8225-03f1ef7f48f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:44.929980 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccd140-9fff-4ad7-8225-03f1ef7f48f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:44.931454 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccd140-9fff-4ad7-8225-03f1ef7f48f9 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:44.940660 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 880/1783] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:48.262752 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb03125-8d9f-4926-8883-560816d3cc2a service cinder] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:48.264272 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6bb03125-8d9f-4926-8883-560816d3cc2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:48.275662 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb03125-8d9f-4926-8883-560816d3cc2a service cinder] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:48.279614 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8931d-11f1-473e-945f-e0eaa371e343 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:48.280212 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8931d-11f1-473e-945f-e0eaa371e343 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:48.280488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8931d-11f1-473e-945f-e0eaa371e343 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:48.280756 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8931d-11f1-473e-945f-e0eaa371e343 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:48.284161 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-0cb8931d-11f1-473e-945f-e0eaa371e343 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:48.356375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb8931d-11f1-473e-945f-e0eaa371e343 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:48.357834 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb8931d-11f1-473e-945f-e0eaa371e343 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:48.368608 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 904/1784] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:54.936715 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=79 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:57.798003 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccd140-9fff-4ad7-8225-03f1ef7f48f9 service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:57.799895 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-49ccd140-9fff-4ad7-8225-03f1ef7f48f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:57.868304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccd140-9fff-4ad7-8225-03f1ef7f48f9 service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:57.873166 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af2a46-5c34-48b3-be16-70aefc734cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:57.873952 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af2a46-5c34-48b3-be16-70aefc734cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:57.874255 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af2a46-5c34-48b3-be16-70aefc734cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:57.874526 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af2a46-5c34-48b3-be16-70aefc734cc8 admin admin] PATH_INFO: `/v3/users/2a5492ab69ee4afbb8c7b0a30818b680` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:57.875770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af2a46-5c34-48b3-be16-70aefc734cc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a5492ab69ee4afbb8c7b0a30818b680)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:57.877756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af2a46-5c34-48b3-be16-70aefc734cc8 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:57.926564 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-99af2a46-5c34-48b3-be16-70aefc734cc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a5492ab69ee4afbb8c7b0a30818b680'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:57.927304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-99af2a46-5c34-48b3-be16-70aefc734cc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a5492ab69ee4afbb8c7b0a30818b680'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:57.928016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-99af2a46-5c34-48b3-be16-70aefc734cc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a5492ab69ee4afbb8c7b0a30818b680'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:57.935286 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-99af2a46-5c34-48b3-be16-70aefc734cc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a5492ab69ee4afbb8c7b0a30818b680'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:57.947333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-99af2a46-5c34-48b3-be16-70aefc734cc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a5492ab69ee4afbb8c7b0a30818b680'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:57.961546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 881/1785] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:12:57 2022] DELETE /identity/v3/users/2a5492ab69ee4afbb8c7b0a30818b680 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:57.967151 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85572cd-3645-47a9-a336-529a48ec7191 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:57.967653 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85572cd-3645-47a9-a336-529a48ec7191 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:57.967741 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85572cd-3645-47a9-a336-529a48ec7191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:57.987365 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-e85572cd-3645-47a9-a336-529a48ec7191 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:57.988683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e85572cd-3645-47a9-a336-529a48ec7191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:58.056352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 905/1786] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:58.066724 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af2a46-5c34-48b3-be16-70aefc734cc8 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:58.068310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-99af2a46-5c34-48b3-be16-70aefc734cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:58.132112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af2a46-5c34-48b3-be16-70aefc734cc8 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:58.136448 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098dc585-9fa0-4a8b-8165-2fe80e6c1f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:58.137076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098dc585-9fa0-4a8b-8165-2fe80e6c1f26 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:58.137358 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098dc585-9fa0-4a8b-8165-2fe80e6c1f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:58.137734 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098dc585-9fa0-4a8b-8165-2fe80e6c1f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:58.157612 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098dc585-9fa0-4a8b-8165-2fe80e6c1f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:58.158878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098dc585-9fa0-4a8b-8165-2fe80e6c1f26 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:58.178228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 882/1787] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:12:58.480808 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85572cd-3645-47a9-a336-529a48ec7191 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:58.482298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e85572cd-3645-47a9-a336-529a48ec7191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:58.494493 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85572cd-3645-47a9-a336-529a48ec7191 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:58.499833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c433b3a-5e78-4031-81d3-ceb555a44456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:58.500484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c433b3a-5e78-4031-81d3-ceb555a44456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:58.500767 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c433b3a-5e78-4031-81d3-ceb555a44456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:58.501046 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c433b3a-5e78-4031-81d3-ceb555a44456 admin admin] PATH_INFO: `/v3/projects/a6b8078cef194f7a9875eca797050074` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:58.502928 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c433b3a-5e78-4031-81d3-ceb555a44456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6b8078cef194f7a9875eca797050074)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:58.504937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c433b3a-5e78-4031-81d3-ceb555a44456 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:58.542009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:58.559011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-7c433b3a-5e78-4031-81d3-ceb555a44456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6b8078cef194f7a9875eca797050074', 'request_id': 'req-7c433b3a-5e78-4031-81d3-ceb555a44456'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:58.564864 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-7c433b3a-5e78-4031-81d3-ceb555a44456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6b8078cef194f7a9875eca797050074', 'request_id': 'req-7c433b3a-5e78-4031-81d3-ceb555a44456'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:58.574607 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-7c433b3a-5e78-4031-81d3-ceb555a44456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6b8078cef194f7a9875eca797050074', 'request_id': 'req-7c433b3a-5e78-4031-81d3-ceb555a44456'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:58.588721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-7c433b3a-5e78-4031-81d3-ceb555a44456 admin admin] The token cache is being invalidate because project a6b8078cef194f7a9875eca797050074 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:12:58.589014 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-7c433b3a-5e78-4031-81d3-ceb555a44456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:58.589720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-7c433b3a-5e78-4031-81d3-ceb555a44456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:58.591602 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 906/1788] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:12:58 2022] DELETE /identity/v3/projects/a6b8078cef194f7a9875eca797050074 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:58.595526 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098dc585-9fa0-4a8b-8165-2fe80e6c1f26 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:58.606358 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098dc585-9fa0-4a8b-8165-2fe80e6c1f26 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:58.629236 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:58.670536 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a593c-4408-4016-bd85-28ee503a4ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:58.671176 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a593c-4408-4016-bd85-28ee503a4ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:58.671489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a593c-4408-4016-bd85-28ee503a4ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:58.671767 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a593c-4408-4016-bd85-28ee503a4ce8 admin admin] PATH_INFO: `/v3/users/d4ec25c162eb481b846b60a41683e7c5` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:58.672931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a593c-4408-4016-bd85-28ee503a4ce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4ec25c162eb481b846b60a41683e7c5)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:58.674885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a593c-4408-4016-bd85-28ee503a4ce8 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:58.726846 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-a04a593c-4408-4016-bd85-28ee503a4ce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4ec25c162eb481b846b60a41683e7c5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:58.727595 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-a04a593c-4408-4016-bd85-28ee503a4ce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4ec25c162eb481b846b60a41683e7c5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:58.728462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-a04a593c-4408-4016-bd85-28ee503a4ce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4ec25c162eb481b846b60a41683e7c5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:58.735467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-a04a593c-4408-4016-bd85-28ee503a4ce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4ec25c162eb481b846b60a41683e7c5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:58.747673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-a04a593c-4408-4016-bd85-28ee503a4ce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4ec25c162eb481b846b60a41683e7c5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:58.761143 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 883/1789] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:12:58 2022] DELETE /identity/v3/users/d4ec25c162eb481b846b60a41683e7c5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:59.050924 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c433b3a-5e78-4031-81d3-ceb555a44456 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:59.053608 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-7c433b3a-5e78-4031-81d3-ceb555a44456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:59.068845 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c433b3a-5e78-4031-81d3-ceb555a44456 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:59.134070 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823536fd-e97e-4f2a-b24b-69c8ad616f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:59.134930 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823536fd-e97e-4f2a-b24b-69c8ad616f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:59.135332 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823536fd-e97e-4f2a-b24b-69c8ad616f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:59.135664 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823536fd-e97e-4f2a-b24b-69c8ad616f86 admin admin] PATH_INFO: `/v3/projects/17b127e5f60a4b5ba93243a42b6f9c29` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:59.136931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823536fd-e97e-4f2a-b24b-69c8ad616f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17b127e5f60a4b5ba93243a42b6f9c29)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:59.139213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823536fd-e97e-4f2a-b24b-69c8ad616f86 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:59.198924 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-823536fd-e97e-4f2a-b24b-69c8ad616f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17b127e5f60a4b5ba93243a42b6f9c29', 'request_id': 'req-823536fd-e97e-4f2a-b24b-69c8ad616f86'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:59.205232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-823536fd-e97e-4f2a-b24b-69c8ad616f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17b127e5f60a4b5ba93243a42b6f9c29', 'request_id': 'req-823536fd-e97e-4f2a-b24b-69c8ad616f86'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:59.215901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-823536fd-e97e-4f2a-b24b-69c8ad616f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17b127e5f60a4b5ba93243a42b6f9c29', 'request_id': 'req-823536fd-e97e-4f2a-b24b-69c8ad616f86'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:59.227458 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-823536fd-e97e-4f2a-b24b-69c8ad616f86 admin admin] The token cache is being invalidate because project 17b127e5f60a4b5ba93243a42b6f9c29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:12:59.227754 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-823536fd-e97e-4f2a-b24b-69c8ad616f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:59.228556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-823536fd-e97e-4f2a-b24b-69c8ad616f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:12:59.230269 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 907/1790] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:12:59 2022] DELETE /identity/v3/projects/17b127e5f60a4b5ba93243a42b6f9c29 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:59.236300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07115815-46f8-4e4d-89d7-bde169cfdd5b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:59.236737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07115815-46f8-4e4d-89d7-bde169cfdd5b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:59.236953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07115815-46f8-4e4d-89d7-bde169cfdd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:59.256671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-07115815-46f8-4e4d-89d7-bde169cfdd5b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:59.257857 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-07115815-46f8-4e4d-89d7-bde169cfdd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:59.335520 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 884/1791] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:59.345788 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c5775-e7ae-4b9c-a383-b8e8a420d244 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:59.346390 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c5775-e7ae-4b9c-a383-b8e8a420d244 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:59.346873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c5775-e7ae-4b9c-a383-b8e8a420d244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:59.380585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-531c5775-e7ae-4b9c-a383-b8e8a420d244 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:59.381902 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-531c5775-e7ae-4b9c-a383-b8e8a420d244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:59.397489 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 908/1792] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:59.401771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07115815-46f8-4e4d-89d7-bde169cfdd5b None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:59.413731 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07115815-46f8-4e4d-89d7-bde169cfdd5b None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:59.419887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2183e-97ee-4d1e-aeb3-5dbb3a2a05b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:59.420714 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2183e-97ee-4d1e-aeb3-5dbb3a2a05b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:59.421056 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2183e-97ee-4d1e-aeb3-5dbb3a2a05b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:59.421396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2183e-97ee-4d1e-aeb3-5dbb3a2a05b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:59.422129 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2183e-97ee-4d1e-aeb3-5dbb3a2a05b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:12:59.422615 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2183e-97ee-4d1e-aeb3-5dbb3a2a05b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:59.424341 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2183e-97ee-4d1e-aeb3-5dbb3a2a05b8 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:59.438362 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 885/1793] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:12:59.443355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c5775-e7ae-4b9c-a383-b8e8a420d244 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:59.453811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c5775-e7ae-4b9c-a383-b8e8a420d244 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:59.459950 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac033e-fa6a-4a59-a2f6-522e66c09249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:59.460576 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac033e-fa6a-4a59-a2f6-522e66c09249 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:59.460858 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac033e-fa6a-4a59-a2f6-522e66c09249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:59.461136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac033e-fa6a-4a59-a2f6-522e66c09249 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:59.461778 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac033e-fa6a-4a59-a2f6-522e66c09249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:59.463937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac033e-fa6a-4a59-a2f6-522e66c09249 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:59.493559 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 909/1794] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:59 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:12:59.497917 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2183e-97ee-4d1e-aeb3-5dbb3a2a05b8 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:59.508397 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2183e-97ee-4d1e-aeb3-5dbb3a2a05b8 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:59.547021 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:59.595801 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c104fa4-421a-4ba8-a2be-462d089d5a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:59.596440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c104fa4-421a-4ba8-a2be-462d089d5a29 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:59.596693 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c104fa4-421a-4ba8-a2be-462d089d5a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:59.596992 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c104fa4-421a-4ba8-a2be-462d089d5a29 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:59.597666 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c104fa4-421a-4ba8-a2be-462d089d5a29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:59.599715 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c104fa4-421a-4ba8-a2be-462d089d5a29 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:59.626949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 886/1795] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:12:59 2022] POST /identity/v3/users => generated 291 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:12:59.630816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac033e-fa6a-4a59-a2f6-522e66c09249 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:59.642472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac033e-fa6a-4a59-a2f6-522e66c09249 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:59.647331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164542f1-7271-413f-9706-ab2bba29e48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:59.648006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164542f1-7271-413f-9706-ab2bba29e48a admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:59.648283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164542f1-7271-413f-9706-ab2bba29e48a admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:59.648557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164542f1-7271-413f-9706-ab2bba29e48a admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:59.649241 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164542f1-7271-413f-9706-ab2bba29e48a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:59.652049 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164542f1-7271-413f-9706-ab2bba29e48a admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:59.664832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 910/1796] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:59.669124 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c104fa4-421a-4ba8-a2be-462d089d5a29 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:59.679123 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c104fa4-421a-4ba8-a2be-462d089d5a29 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:59.686813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d76b5-1d1c-422c-8cdd-7fec82948982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:59.687462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d76b5-1d1c-422c-8cdd-7fec82948982 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:59.687746 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d76b5-1d1c-422c-8cdd-7fec82948982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:59.688017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d76b5-1d1c-422c-8cdd-7fec82948982 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:59.688670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d76b5-1d1c-422c-8cdd-7fec82948982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:59.690286 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d76b5-1d1c-422c-8cdd-7fec82948982 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:59.702231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 887/1797] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:12:59.706618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164542f1-7271-413f-9706-ab2bba29e48a admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:59.717488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164542f1-7271-413f-9706-ab2bba29e48a admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:59.722210 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737de7c3-99ff-4fbe-8541-0e0196b53033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:59.722870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737de7c3-99ff-4fbe-8541-0e0196b53033 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:59.723153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737de7c3-99ff-4fbe-8541-0e0196b53033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:59.723458 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737de7c3-99ff-4fbe-8541-0e0196b53033 admin admin] PATH_INFO: `/v3/projects/43afb0442c3445548050cc02bfcf44ae/users/44bc08892b4e491c808fce95548b7822/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:59.725744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:12:59.742336 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737de7c3-99ff-4fbe-8541-0e0196b53033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43afb0442c3445548050cc02bfcf44ae, user_id=44bc08892b4e491c808fce95548b7822, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:59.746479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737de7c3-99ff-4fbe-8541-0e0196b53033 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:59.761643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 911/1798] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:12:59 2022] PUT /identity/v3/projects/43afb0442c3445548050cc02bfcf44ae/users/44bc08892b4e491c808fce95548b7822/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:12:59.773191 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78947d6-5bb0-4f43-abb6-7a7373bd2a16 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:59.773508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78947d6-5bb0-4f43-abb6-7a7373bd2a16 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:59.773781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78947d6-5bb0-4f43-abb6-7a7373bd2a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:59.803686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-f78947d6-5bb0-4f43-abb6-7a7373bd2a16 None None] MFA Rules not processed for user `44bc08892b4e491c808fce95548b7822`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:59.804958 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f78947d6-5bb0-4f43-abb6-7a7373bd2a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:59.867181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abce13-2fba-46be-bdf2-91585386ffc2 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:59.867655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abce13-2fba-46be-bdf2-91585386ffc2 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:59.867655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abce13-2fba-46be-bdf2-91585386ffc2 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:59.869212 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 912/1799] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 23 07:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:12:59.877554 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42391b2-7eb1-421e-9aee-1c5c9aa72657 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:59.879161 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42391b2-7eb1-421e-9aee-1c5c9aa72657 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:59.879762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42391b2-7eb1-421e-9aee-1c5c9aa72657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:59.882867 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 888/1800] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:12:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:59.888195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78947d6-5bb0-4f43-abb6-7a7373bd2a16 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:59.903097 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-a42391b2-7eb1-421e-9aee-1c5c9aa72657 None None] MFA Rules not processed for user `a8d2cd93042c4583b63afe629d47fce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:12:59.904284 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a42391b2-7eb1-421e-9aee-1c5c9aa72657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:12:59.968128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78947d6-5bb0-4f43-abb6-7a7373bd2a16 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:12:59.972716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb612d51-5220-476d-bb66-b0a11cf5d6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:12:59.973368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb612d51-5220-476d-bb66-b0a11cf5d6ad admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:12:59.973644 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb612d51-5220-476d-bb66-b0a11cf5d6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:12:59.973916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb612d51-5220-476d-bb66-b0a11cf5d6ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:12:59.974556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb612d51-5220-476d-bb66-b0a11cf5d6ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:12:59.976521 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb612d51-5220-476d-bb66-b0a11cf5d6ad admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:12:59.977815 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 913/1801] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 23 07:12:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:12:59.991004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42391b2-7eb1-421e-9aee-1c5c9aa72657 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:12:59.992443 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a42391b2-7eb1-421e-9aee-1c5c9aa72657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:00.006897 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 889/1802] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:12:59 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:00.009048 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42391b2-7eb1-421e-9aee-1c5c9aa72657 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:00.010611 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb612d51-5220-476d-bb66-b0a11cf5d6ad admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:00.040813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:00.081675 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb612d51-5220-476d-bb66-b0a11cf5d6ad admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:00.085119 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d430d3f-a96c-4f4c-9890-3bcb94ee0861 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:00.085750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d430d3f-a96c-4f4c-9890-3bcb94ee0861 service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:00.086030 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d430d3f-a96c-4f4c-9890-3bcb94ee0861 service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:00.086239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21b36c-05db-454b-ae48-589e57e973d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:00.086378 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d430d3f-a96c-4f4c-9890-3bcb94ee0861 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:00.086848 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21b36c-05db-454b-ae48-589e57e973d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:00.087129 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21b36c-05db-454b-ae48-589e57e973d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:00.087404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21b36c-05db-454b-ae48-589e57e973d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:00.088219 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21b36c-05db-454b-ae48-589e57e973d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:00.091606 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21b36c-05db-454b-ae48-589e57e973d3 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:00.096209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d430d3f-a96c-4f4c-9890-3bcb94ee0861 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:00.097806 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d430d3f-a96c-4f4c-9890-3bcb94ee0861 service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:00.109769 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 914/1803] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:12:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:13:00.121005 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 890/1804] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:00 2022] POST /identity/v3/users => generated 298 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:00.122116 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21b36c-05db-454b-ae48-589e57e973d3 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:00.131668 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21b36c-05db-454b-ae48-589e57e973d3 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:00.136440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d31df5-efd0-463b-a919-9cfed95a2f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:00.137037 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d31df5-efd0-463b-a919-9cfed95a2f8d admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:00.137313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d31df5-efd0-463b-a919-9cfed95a2f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:00.137584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d31df5-efd0-463b-a919-9cfed95a2f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:00.138221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d31df5-efd0-463b-a919-9cfed95a2f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:00.139857 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d31df5-efd0-463b-a919-9cfed95a2f8d admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:00.150966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 891/1805] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:00.155146 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d430d3f-a96c-4f4c-9890-3bcb94ee0861 service nova] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:00.166022 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d430d3f-a96c-4f4c-9890-3bcb94ee0861 service nova] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:00.171371 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d009c-c092-41b4-a24d-36ba217e316b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:00.172054 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d009c-c092-41b4-a24d-36ba217e316b admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:00.172333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d009c-c092-41b4-a24d-36ba217e316b admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:00.172610 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d009c-c092-41b4-a24d-36ba217e316b admin admin] PATH_INFO: `/v3/projects/e71bbfb3976b4be4afd710c340f34c73/users/2c9e3abd9bbe4ded98fc1bd049200a46/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:00.189900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d009c-c092-41b4-a24d-36ba217e316b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e71bbfb3976b4be4afd710c340f34c73, user_id=2c9e3abd9bbe4ded98fc1bd049200a46, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:00.192655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d009c-c092-41b4-a24d-36ba217e316b admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:00.206993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 915/1806] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:00 2022] PUT /identity/v3/projects/e71bbfb3976b4be4afd710c340f34c73/users/2c9e3abd9bbe4ded98fc1bd049200a46/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:00.211388 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d31df5-efd0-463b-a919-9cfed95a2f8d admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:00.226151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d31df5-efd0-463b-a919-9cfed95a2f8d admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:00.284987 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f27508-d349-4f15-8540-2efde682a548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:00.285607 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f27508-d349-4f15-8540-2efde682a548 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:00.285884 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f27508-d349-4f15-8540-2efde682a548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:00.286150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f27508-d349-4f15-8540-2efde682a548 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:00.286790 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f27508-d349-4f15-8540-2efde682a548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:00.288417 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f27508-d349-4f15-8540-2efde682a548 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:00.298787 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 892/1807] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:00.303180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d009c-c092-41b4-a24d-36ba217e316b admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:00.314314 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d009c-c092-41b4-a24d-36ba217e316b admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:00.320214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd05d89-bae8-4362-8c2a-c20eaf06c5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:00.320869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd05d89-bae8-4362-8c2a-c20eaf06c5fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:00.321148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd05d89-bae8-4362-8c2a-c20eaf06c5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:00.321420 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd05d89-bae8-4362-8c2a-c20eaf06c5fe admin admin] PATH_INFO: `/v3/projects/e71bbfb3976b4be4afd710c340f34c73/users/2c9e3abd9bbe4ded98fc1bd049200a46/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:00.324261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd05d89-bae8-4362-8c2a-c20eaf06c5fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e71bbfb3976b4be4afd710c340f34c73, user_id=2c9e3abd9bbe4ded98fc1bd049200a46, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:00.326938 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd05d89-bae8-4362-8c2a-c20eaf06c5fe admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:00.338119 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 916/1808] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:00 2022] PUT /identity/v3/projects/e71bbfb3976b4be4afd710c340f34c73/users/2c9e3abd9bbe4ded98fc1bd049200a46/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:00.348599 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710e93d-457c-419a-82cd-3d4a02c38dd2 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:00.348884 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710e93d-457c-419a-82cd-3d4a02c38dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:00.349155 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710e93d-457c-419a-82cd-3d4a02c38dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:00.379553 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-a710e93d-457c-419a-82cd-3d4a02c38dd2 None None] MFA Rules not processed for user `2c9e3abd9bbe4ded98fc1bd049200a46`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:00.380799 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-a710e93d-457c-419a-82cd-3d4a02c38dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:00.449774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 893/1809] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:00.465269 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd05d89-bae8-4362-8c2a-c20eaf06c5fe admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:00.467170 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6dd05d89-bae8-4362-8c2a-c20eaf06c5fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:00.495360 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:00.538105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd05d89-bae8-4362-8c2a-c20eaf06c5fe admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:00.541998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec9a54-adc6-412d-8909-6cbb31769f44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:00.542612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec9a54-adc6-412d-8909-6cbb31769f44 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:00.542891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec9a54-adc6-412d-8909-6cbb31769f44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:00.543159 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec9a54-adc6-412d-8909-6cbb31769f44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:00.552988 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec9a54-adc6-412d-8909-6cbb31769f44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:00.554408 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec9a54-adc6-412d-8909-6cbb31769f44 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:00.564079 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 917/1810] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:00.678111 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710e93d-457c-419a-82cd-3d4a02c38dd2 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:00.688010 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710e93d-457c-419a-82cd-3d4a02c38dd2 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:00.691642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55284f2c-69a1-4849-bf2d-c3aa57ea1015 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:00.692251 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55284f2c-69a1-4849-bf2d-c3aa57ea1015 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:00.692528 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55284f2c-69a1-4849-bf2d-c3aa57ea1015 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:00.692792 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55284f2c-69a1-4849-bf2d-c3aa57ea1015 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:00.715277 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:00.755883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55284f2c-69a1-4849-bf2d-c3aa57ea1015 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:00.757346 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55284f2c-69a1-4849-bf2d-c3aa57ea1015 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:00.780682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 894/1811] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:03.552308 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec9a54-adc6-412d-8909-6cbb31769f44 service cinder] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:03.554267 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-65ec9a54-adc6-412d-8909-6cbb31769f44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:03.679211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec9a54-adc6-412d-8909-6cbb31769f44 service cinder] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:03.693085 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dce48-9273-48f6-be34-5b471a821de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:03.693768 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dce48-9273-48f6-be34-5b471a821de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:03.693955 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dce48-9273-48f6-be34-5b471a821de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:03.694232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dce48-9273-48f6-be34-5b471a821de8 admin admin] PATH_INFO: `/v3/users/a0e300baac294ebe867407ff20334637` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:03.696056 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814dce48-9273-48f6-be34-5b471a821de8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0e300baac294ebe867407ff20334637)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:03.701224 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814dce48-9273-48f6-be34-5b471a821de8 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:03.810794 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-814dce48-9273-48f6-be34-5b471a821de8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0e300baac294ebe867407ff20334637'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:03.811730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-814dce48-9273-48f6-be34-5b471a821de8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0e300baac294ebe867407ff20334637'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:03.813621 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-814dce48-9273-48f6-be34-5b471a821de8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0e300baac294ebe867407ff20334637'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:03.832464 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-814dce48-9273-48f6-be34-5b471a821de8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0e300baac294ebe867407ff20334637'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:03.864167 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-814dce48-9273-48f6-be34-5b471a821de8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0e300baac294ebe867407ff20334637'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:03.898311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 918/1812] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:03 2022] DELETE /identity/v3/users/a0e300baac294ebe867407ff20334637 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:03.898413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c341dbe-df73-47ae-87af-89e017f00343 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:03.898828 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c341dbe-df73-47ae-87af-89e017f00343 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:03.899396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c341dbe-df73-47ae-87af-89e017f00343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:03.964474 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-3c341dbe-df73-47ae-87af-89e017f00343 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:03.966658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3c341dbe-df73-47ae-87af-89e017f00343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:04.062179 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 895/1813] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:04.135091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dce48-9273-48f6-be34-5b471a821de8 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:04.138158 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-814dce48-9273-48f6-be34-5b471a821de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:04.244550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:04.282331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dce48-9273-48f6-be34-5b471a821de8 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:04.294792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832c267-be67-4621-9c3a-e62873a5239f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:04.297214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832c267-be67-4621-9c3a-e62873a5239f service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:04.299608 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832c267-be67-4621-9c3a-e62873a5239f service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:04.300900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832c267-be67-4621-9c3a-e62873a5239f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:04.350044 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b832c267-be67-4621-9c3a-e62873a5239f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:04.351706 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b832c267-be67-4621-9c3a-e62873a5239f service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:04.364484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 919/1814] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:04.668967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c341dbe-df73-47ae-87af-89e017f00343 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:04.670653 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3c341dbe-df73-47ae-87af-89e017f00343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:04.685862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c341dbe-df73-47ae-87af-89e017f00343 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:04.689804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbceb46-2c8e-4951-b423-b88dbe0dc895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:04.690616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbceb46-2c8e-4951-b423-b88dbe0dc895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:04.690961 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbceb46-2c8e-4951-b423-b88dbe0dc895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:04.691357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbceb46-2c8e-4951-b423-b88dbe0dc895 admin admin] PATH_INFO: `/v3/users/44bc08892b4e491c808fce95548b7822` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:04.692843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbceb46-2c8e-4951-b423-b88dbe0dc895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44bc08892b4e491c808fce95548b7822)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:04.694990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbceb46-2c8e-4951-b423-b88dbe0dc895 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:04.752614 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-7bbceb46-2c8e-4951-b423-b88dbe0dc895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44bc08892b4e491c808fce95548b7822'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:04.753650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-7bbceb46-2c8e-4951-b423-b88dbe0dc895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44bc08892b4e491c808fce95548b7822'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:04.754423 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-7bbceb46-2c8e-4951-b423-b88dbe0dc895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44bc08892b4e491c808fce95548b7822'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:04.762379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-7bbceb46-2c8e-4951-b423-b88dbe0dc895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44bc08892b4e491c808fce95548b7822'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:04.775728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-7bbceb46-2c8e-4951-b423-b88dbe0dc895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44bc08892b4e491c808fce95548b7822'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:04.790909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 896/1815] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:04 2022] DELETE /identity/v3/users/44bc08892b4e491c808fce95548b7822 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:04.795717 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e32e3-d452-49e9-bc1a-a1af8f27f975 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:04.796344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e32e3-d452-49e9-bc1a-a1af8f27f975 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:04.796344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e32e3-d452-49e9-bc1a-a1af8f27f975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:04.816123 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-516e32e3-d452-49e9-bc1a-a1af8f27f975 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:04.817519 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-516e32e3-d452-49e9-bc1a-a1af8f27f975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:04.883296 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 920/1816] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:04.893516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbceb46-2c8e-4951-b423-b88dbe0dc895 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:04.895453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-7bbceb46-2c8e-4951-b423-b88dbe0dc895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:04.950482 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:04.980317 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbceb46-2c8e-4951-b423-b88dbe0dc895 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:04.984511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4cb90-0b38-4541-822d-60150c8cfbaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:04.985137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4cb90-0b38-4541-822d-60150c8cfbaa service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:04.985788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4cb90-0b38-4541-822d-60150c8cfbaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:04.985788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4cb90-0b38-4541-822d-60150c8cfbaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:04.998249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4cb90-0b38-4541-822d-60150c8cfbaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:05.000039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4cb90-0b38-4541-822d-60150c8cfbaa service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:05.012618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 897/1817] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:05.389007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e32e3-d452-49e9-bc1a-a1af8f27f975 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:05.391872 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-516e32e3-d452-49e9-bc1a-a1af8f27f975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:05.414296 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e32e3-d452-49e9-bc1a-a1af8f27f975 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:05.425952 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4cb90-0b38-4541-822d-60150c8cfbaa service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:05.427976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-65d4cb90-0b38-4541-822d-60150c8cfbaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:05.433922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b160b-2cc1-47f4-81cc-1b9f6ba3a826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:05.434584 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b160b-2cc1-47f4-81cc-1b9f6ba3a826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:05.434863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b160b-2cc1-47f4-81cc-1b9f6ba3a826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:05.435143 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b160b-2cc1-47f4-81cc-1b9f6ba3a826 admin admin] PATH_INFO: `/v3/projects/43afb0442c3445548050cc02bfcf44ae` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:05.436571 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b160b-2cc1-47f4-81cc-1b9f6ba3a826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43afb0442c3445548050cc02bfcf44ae)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:05.438624 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b160b-2cc1-47f4-81cc-1b9f6ba3a826 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:05.443077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4cb90-0b38-4541-822d-60150c8cfbaa service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:05.448716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9a4a2-c5ec-4706-becf-7cd0207cdac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:05.464803 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9a4a2-c5ec-4706-becf-7cd0207cdac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:05.465100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9a4a2-c5ec-4706-becf-7cd0207cdac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:05.465381 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9a4a2-c5ec-4706-becf-7cd0207cdac7 admin admin] PATH_INFO: `/v3/projects/771b3076a9ac4c44819c559859a1add3` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:05.478142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9a4a2-c5ec-4706-becf-7cd0207cdac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=771b3076a9ac4c44819c559859a1add3)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:05.481003 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9a4a2-c5ec-4706-becf-7cd0207cdac7 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:05.507280 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-930b160b-2cc1-47f4-81cc-1b9f6ba3a826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43afb0442c3445548050cc02bfcf44ae', 'request_id': 'req-930b160b-2cc1-47f4-81cc-1b9f6ba3a826'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:05.515633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-930b160b-2cc1-47f4-81cc-1b9f6ba3a826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43afb0442c3445548050cc02bfcf44ae', 'request_id': 'req-930b160b-2cc1-47f4-81cc-1b9f6ba3a826'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:05.529430 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-930b160b-2cc1-47f4-81cc-1b9f6ba3a826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43afb0442c3445548050cc02bfcf44ae', 'request_id': 'req-930b160b-2cc1-47f4-81cc-1b9f6ba3a826'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:05.544438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-930b160b-2cc1-47f4-81cc-1b9f6ba3a826 admin admin] The token cache is being invalidate because project 43afb0442c3445548050cc02bfcf44ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:05.544735 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-930b160b-2cc1-47f4-81cc-1b9f6ba3a826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:05.545438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-930b160b-2cc1-47f4-81cc-1b9f6ba3a826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:05.547166 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 921/1818] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:05 2022] DELETE /identity/v3/projects/43afb0442c3445548050cc02bfcf44ae => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:05.549979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-0cb9a4a2-c5ec-4706-becf-7cd0207cdac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '771b3076a9ac4c44819c559859a1add3', 'request_id': 'req-0cb9a4a2-c5ec-4706-becf-7cd0207cdac7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:05.551214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b160b-2cc1-47f4-81cc-1b9f6ba3a826 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:05.552675 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-930b160b-2cc1-47f4-81cc-1b9f6ba3a826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:05.556594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-0cb9a4a2-c5ec-4706-becf-7cd0207cdac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '771b3076a9ac4c44819c559859a1add3', 'request_id': 'req-0cb9a4a2-c5ec-4706-becf-7cd0207cdac7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:05.570157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-0cb9a4a2-c5ec-4706-becf-7cd0207cdac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '771b3076a9ac4c44819c559859a1add3', 'request_id': 'req-0cb9a4a2-c5ec-4706-becf-7cd0207cdac7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:05.585677 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-0cb9a4a2-c5ec-4706-becf-7cd0207cdac7 admin admin] The token cache is being invalidate because project 771b3076a9ac4c44819c559859a1add3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:05.585967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-0cb9a4a2-c5ec-4706-becf-7cd0207cdac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:05.586703 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-0cb9a4a2-c5ec-4706-becf-7cd0207cdac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:05.588836 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 898/1819] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:05 2022] DELETE /identity/v3/projects/771b3076a9ac4c44819c559859a1add3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:05.593342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7629b89-62bc-4c74-8f20-96e00165d2bd None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:05.593623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7629b89-62bc-4c74-8f20-96e00165d2bd None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:05.593894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7629b89-62bc-4c74-8f20-96e00165d2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:05.617027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-f7629b89-62bc-4c74-8f20-96e00165d2bd None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:05.618664 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f7629b89-62bc-4c74-8f20-96e00165d2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:05.626569 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b160b-2cc1-47f4-81cc-1b9f6ba3a826 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:05.640882 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:05.701169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588de573-ed25-458c-a6dd-8599ba4c7eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:05.702219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588de573-ed25-458c-a6dd-8599ba4c7eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:05.702750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588de573-ed25-458c-a6dd-8599ba4c7eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:05.703000 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588de573-ed25-458c-a6dd-8599ba4c7eea admin admin] PATH_INFO: `/v3/users/2c9e3abd9bbe4ded98fc1bd049200a46` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:05.704404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588de573-ed25-458c-a6dd-8599ba4c7eea admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c9e3abd9bbe4ded98fc1bd049200a46)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:05.706973 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588de573-ed25-458c-a6dd-8599ba4c7eea admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:05.717218 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 899/1820] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:05.724202 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957ca3d-d74e-4bb9-969a-1cf5885de16b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:05.724575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957ca3d-d74e-4bb9-969a-1cf5885de16b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:05.724919 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957ca3d-d74e-4bb9-969a-1cf5885de16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:05.745443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-7957ca3d-d74e-4bb9-969a-1cf5885de16b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:05.746733 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-7957ca3d-d74e-4bb9-969a-1cf5885de16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:05.754760 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 900/1821] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:05.755049 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-588de573-ed25-458c-a6dd-8599ba4c7eea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c9e3abd9bbe4ded98fc1bd049200a46'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:05.755795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-588de573-ed25-458c-a6dd-8599ba4c7eea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c9e3abd9bbe4ded98fc1bd049200a46'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:05.756468 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-588de573-ed25-458c-a6dd-8599ba4c7eea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c9e3abd9bbe4ded98fc1bd049200a46'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:05.758859 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957ca3d-d74e-4bb9-969a-1cf5885de16b None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:05.763951 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-588de573-ed25-458c-a6dd-8599ba4c7eea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c9e3abd9bbe4ded98fc1bd049200a46'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:05.768542 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957ca3d-d74e-4bb9-969a-1cf5885de16b None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:05.773118 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5d408-f22b-4227-a31b-286c51c4795a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:05.773705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5d408-f22b-4227-a31b-286c51c4795a admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:05.774011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5d408-f22b-4227-a31b-286c51c4795a admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:05.774276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5d408-f22b-4227-a31b-286c51c4795a admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:05.774957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5d408-f22b-4227-a31b-286c51c4795a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:05.775271 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5d408-f22b-4227-a31b-286c51c4795a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:05.775937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-588de573-ed25-458c-a6dd-8599ba4c7eea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c9e3abd9bbe4ded98fc1bd049200a46'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:05.776925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5d408-f22b-4227-a31b-286c51c4795a admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:05.796216 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 901/1822] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:13:05.796372 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 922/1823] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:05 2022] DELETE /identity/v3/users/2c9e3abd9bbe4ded98fc1bd049200a46 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:05.797142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5d408-f22b-4227-a31b-286c51c4795a admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:05.798804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-86a5d408-f22b-4227-a31b-286c51c4795a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:05.811755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5d408-f22b-4227-a31b-286c51c4795a admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:05.851347 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:05.872632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78370af-5588-4b70-9ed4-fd2f6095b78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:05.873254 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78370af-5588-4b70-9ed4-fd2f6095b78c admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:05.873537 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78370af-5588-4b70-9ed4-fd2f6095b78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:05.873810 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78370af-5588-4b70-9ed4-fd2f6095b78c admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:05.874443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78370af-5588-4b70-9ed4-fd2f6095b78c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:05.876441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78370af-5588-4b70-9ed4-fd2f6095b78c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:05.906144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 902/1824] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:05 2022] POST /identity/v3/projects => generated 403 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:05.910487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588de573-ed25-458c-a6dd-8599ba4c7eea admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:05.921550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588de573-ed25-458c-a6dd-8599ba4c7eea admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:05.983740 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d7597-3454-4fb5-8c97-14624c6fa8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:05.984358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d7597-3454-4fb5-8c97-14624c6fa8a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:05.984637 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d7597-3454-4fb5-8c97-14624c6fa8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:05.984912 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d7597-3454-4fb5-8c97-14624c6fa8a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:05.985558 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d7597-3454-4fb5-8c97-14624c6fa8a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:05.987639 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d7597-3454-4fb5-8c97-14624c6fa8a1 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:06.011820 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 923/1825] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:05 2022] POST /identity/v3/users => generated 302 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:06.016014 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78370af-5588-4b70-9ed4-fd2f6095b78c admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:06.026672 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78370af-5588-4b70-9ed4-fd2f6095b78c admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:06.031436 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308dcc9f-78ed-4cf2-97c6-ed924e0a2a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:06.032093 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dcc9f-78ed-4cf2-97c6-ed924e0a2a35 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.032373 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dcc9f-78ed-4cf2-97c6-ed924e0a2a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.032646 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dcc9f-78ed-4cf2-97c6-ed924e0a2a35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.033330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308dcc9f-78ed-4cf2-97c6-ed924e0a2a35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:06.034947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308dcc9f-78ed-4cf2-97c6-ed924e0a2a35 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:06.045762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 903/1826] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:06.049993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d7597-3454-4fb5-8c97-14624c6fa8a1 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:06.060646 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d7597-3454-4fb5-8c97-14624c6fa8a1 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:06.065923 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f8487-0e3e-4f84-bd19-0c859bede755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:06.066613 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f8487-0e3e-4f84-bd19-0c859bede755 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.067048 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f8487-0e3e-4f84-bd19-0c859bede755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.067286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f8487-0e3e-4f84-bd19-0c859bede755 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.068369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f8487-0e3e-4f84-bd19-0c859bede755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:06.069775 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f8487-0e3e-4f84-bd19-0c859bede755 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:06.081683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 924/1827] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:06.086128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308dcc9f-78ed-4cf2-97c6-ed924e0a2a35 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:06.088439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f8487-0e3e-4f84-bd19-0c859bede755 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:06.088645 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-308dcc9f-78ed-4cf2-97c6-ed924e0a2a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:06.105793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308dcc9f-78ed-4cf2-97c6-ed924e0a2a35 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:06.107775 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f8487-0e3e-4f84-bd19-0c859bede755 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:06.111865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1ec48-2bf6-4791-9ef2-805abb01ed50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:06.112609 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1ec48-2bf6-4791-9ef2-805abb01ed50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.112895 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1ec48-2bf6-4791-9ef2-805abb01ed50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.113185 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1ec48-2bf6-4791-9ef2-805abb01ed50 admin admin] PATH_INFO: `/v3/projects/e71bbfb3976b4be4afd710c340f34c73` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.114586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2917228-c72a-44de-a65a-d7339483f803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:06.115148 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e1ec48-2bf6-4791-9ef2-805abb01ed50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e71bbfb3976b4be4afd710c340f34c73)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:06.115813 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2917228-c72a-44de-a65a-d7339483f803 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.116167 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2917228-c72a-44de-a65a-d7339483f803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.116419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2917228-c72a-44de-a65a-d7339483f803 admin admin] PATH_INFO: `/v3/projects/2012102220bf443ca51a95e0a38c77d4/users/c38ef6b1f29f4e07ac607c64f28b0773/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.118013 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e1ec48-2bf6-4791-9ef2-805abb01ed50 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:06.132571 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:06.140317 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2917228-c72a-44de-a65a-d7339483f803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2012102220bf443ca51a95e0a38c77d4, user_id=c38ef6b1f29f4e07ac607c64f28b0773, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:06.143728 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2917228-c72a-44de-a65a-d7339483f803 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:06.161440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 925/1828] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:06 2022] PUT /identity/v3/projects/2012102220bf443ca51a95e0a38c77d4/users/c38ef6b1f29f4e07ac607c64f28b0773/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:06.172207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5ea44-6e3c-4318-bd33-f756c5972d2d None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.172974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5ea44-6e3c-4318-bd33-f756c5972d2d None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.173253 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5ea44-6e3c-4318-bd33-f756c5972d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.184559 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-c9e1ec48-2bf6-4791-9ef2-805abb01ed50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e71bbfb3976b4be4afd710c340f34c73', 'request_id': 'req-c9e1ec48-2bf6-4791-9ef2-805abb01ed50'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:06.193331 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-c9e1ec48-2bf6-4791-9ef2-805abb01ed50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e71bbfb3976b4be4afd710c340f34c73', 'request_id': 'req-c9e1ec48-2bf6-4791-9ef2-805abb01ed50'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:06.210453 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-65d5ea44-6e3c-4318-bd33-f756c5972d2d None None] MFA Rules not processed for user `c38ef6b1f29f4e07ac607c64f28b0773`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:06.211778 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-65d5ea44-6e3c-4318-bd33-f756c5972d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:06.214185 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-c9e1ec48-2bf6-4791-9ef2-805abb01ed50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e71bbfb3976b4be4afd710c340f34c73', 'request_id': 'req-c9e1ec48-2bf6-4791-9ef2-805abb01ed50'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:06.228703 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-c9e1ec48-2bf6-4791-9ef2-805abb01ed50 admin admin] The token cache is being invalidate because project e71bbfb3976b4be4afd710c340f34c73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:06.229111 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-c9e1ec48-2bf6-4791-9ef2-805abb01ed50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:06.229845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-c9e1ec48-2bf6-4791-9ef2-805abb01ed50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:06.231600 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 904/1829] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:06 2022] DELETE /identity/v3/projects/e71bbfb3976b4be4afd710c340f34c73 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:06.236380 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738a304-9c0d-4dd1-9cef-05f0d33647b2 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.236745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738a304-9c0d-4dd1-9cef-05f0d33647b2 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.237082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738a304-9c0d-4dd1-9cef-05f0d33647b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.259178 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-1738a304-9c0d-4dd1-9cef-05f0d33647b2 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:06.262147 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-1738a304-9c0d-4dd1-9cef-05f0d33647b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:06.291519 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 926/1830] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:06 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:06.305100 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5ea44-6e3c-4318-bd33-f756c5972d2d None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:06.308008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-65d5ea44-6e3c-4318-bd33-f756c5972d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:06.343532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 905/1831] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:06.351335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec173dce-03da-494f-b371-21aa339d613f None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.351943 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec173dce-03da-494f-b371-21aa339d613f None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.352413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec173dce-03da-494f-b371-21aa339d613f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.383583 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-ec173dce-03da-494f-b371-21aa339d613f None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:06.386422 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ec173dce-03da-494f-b371-21aa339d613f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:06.395035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5ea44-6e3c-4318-bd33-f756c5972d2d None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:06.398238 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 906/1832] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:06.402470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec173dce-03da-494f-b371-21aa339d613f None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:06.404313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf0fa6-c82d-4ff3-b501-cb69a2142065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:06.405428 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf0fa6-c82d-4ff3-b501-cb69a2142065 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.405428 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf0fa6-c82d-4ff3-b501-cb69a2142065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.405708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf0fa6-c82d-4ff3-b501-cb69a2142065 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.406801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf0fa6-c82d-4ff3-b501-cb69a2142065 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:06.409623 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf0fa6-c82d-4ff3-b501-cb69a2142065 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:06.414178 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec173dce-03da-494f-b371-21aa339d613f None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:06.441257 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5560b-b515-4e1c-8c06-d23460ccdd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:06.441257 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5560b-b515-4e1c-8c06-d23460ccdd50 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.441257 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5560b-b515-4e1c-8c06-d23460ccdd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.441257 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5560b-b515-4e1c-8c06-d23460ccdd50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.441668 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5560b-b515-4e1c-8c06-d23460ccdd50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:06.441668 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5560b-b515-4e1c-8c06-d23460ccdd50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:06.441668 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5560b-b515-4e1c-8c06-d23460ccdd50 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:06.453427 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 907/1833] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:06.459115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 927/1834] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:06 2022] POST /identity/v3/projects => generated 405 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:06.459290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5560b-b515-4e1c-8c06-d23460ccdd50 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:06.460470 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf0fa6-c82d-4ff3-b501-cb69a2142065 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:06.471661 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5560b-b515-4e1c-8c06-d23460ccdd50 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:06.485850 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:06.535700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf0fa6-c82d-4ff3-b501-cb69a2142065 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:06.541327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f947b-7080-47ec-a702-0079fd4caa5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:06.542026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f947b-7080-47ec-a702-0079fd4caa5d admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.542376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f947b-7080-47ec-a702-0079fd4caa5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.542739 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f947b-7080-47ec-a702-0079fd4caa5d admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.543476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f947b-7080-47ec-a702-0079fd4caa5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:06.545488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f947b-7080-47ec-a702-0079fd4caa5d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:06.557077 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:06.563050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420649ea-9691-4d8d-a8de-1141326b7f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:06.566227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420649ea-9691-4d8d-a8de-1141326b7f27 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.566227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420649ea-9691-4d8d-a8de-1141326b7f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.566227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420649ea-9691-4d8d-a8de-1141326b7f27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.566227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420649ea-9691-4d8d-a8de-1141326b7f27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:06.566774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420649ea-9691-4d8d-a8de-1141326b7f27 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:06.571877 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 928/1835] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:06 2022] POST /identity/v3/users => generated 309 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:06.576679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f947b-7080-47ec-a702-0079fd4caa5d admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:06.587222 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f947b-7080-47ec-a702-0079fd4caa5d admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:06.593087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f23fa-9e4f-42bd-bd91-6d8acdea49ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:06.593724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f23fa-9e4f-42bd-bd91-6d8acdea49ff admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.593999 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f23fa-9e4f-42bd-bd91-6d8acdea49ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.594275 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f23fa-9e4f-42bd-bd91-6d8acdea49ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.594924 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f23fa-9e4f-42bd-bd91-6d8acdea49ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:06.596565 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f23fa-9e4f-42bd-bd91-6d8acdea49ff admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:06.602678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 908/1836] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:06 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:06.603837 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420649ea-9691-4d8d-a8de-1141326b7f27 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:06.611204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 929/1837] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:06.612436 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f23fa-9e4f-42bd-bd91-6d8acdea49ff admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:06.622757 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f23fa-9e4f-42bd-bd91-6d8acdea49ff admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:06.665719 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420649ea-9691-4d8d-a8de-1141326b7f27 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:06.671085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bf177-aafa-4c42-b031-f008665f64be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:06.671740 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bf177-aafa-4c42-b031-f008665f64be admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.672016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bf177-aafa-4c42-b031-f008665f64be admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.672287 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bf177-aafa-4c42-b031-f008665f64be admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.672915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446bf177-aafa-4c42-b031-f008665f64be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:06.674991 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446bf177-aafa-4c42-b031-f008665f64be admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:06.687376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a134fb4-f106-4938-a38e-9cca55d1188e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:06.688098 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a134fb4-f106-4938-a38e-9cca55d1188e admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.688375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a134fb4-f106-4938-a38e-9cca55d1188e admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.688652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a134fb4-f106-4938-a38e-9cca55d1188e admin admin] PATH_INFO: `/v3/projects/012e36787dcf4daf966d29119dc8f4c7/users/3d4089961c324c7483382f64a943891d/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.699251 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 909/1838] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:06 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:06.703832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bf177-aafa-4c42-b031-f008665f64be admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:06.709698 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a134fb4-f106-4938-a38e-9cca55d1188e admin admin] RBAC: Authorizing `identity:create_grant(project_id=012e36787dcf4daf966d29119dc8f4c7, user_id=3d4089961c324c7483382f64a943891d, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:06.712749 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a134fb4-f106-4938-a38e-9cca55d1188e admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:06.716213 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bf177-aafa-4c42-b031-f008665f64be admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:06.722098 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca4098-a69e-4703-a9f0-4ceccfdd34fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:06.722909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca4098-a69e-4703-a9f0-4ceccfdd34fe admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.723196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca4098-a69e-4703-a9f0-4ceccfdd34fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.723751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca4098-a69e-4703-a9f0-4ceccfdd34fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.724409 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca4098-a69e-4703-a9f0-4ceccfdd34fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:06.726059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca4098-a69e-4703-a9f0-4ceccfdd34fe admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:06.729921 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 930/1839] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:06 2022] PUT /identity/v3/projects/012e36787dcf4daf966d29119dc8f4c7/users/3d4089961c324c7483382f64a943891d/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:06.733844 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a134fb4-f106-4938-a38e-9cca55d1188e admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:06.745466 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 910/1840] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:06.746913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca4098-a69e-4703-a9f0-4ceccfdd34fe admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:06.756492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca4098-a69e-4703-a9f0-4ceccfdd34fe admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:06.797557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a134fb4-f106-4938-a38e-9cca55d1188e admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:06.802216 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f8eeb-beed-46f9-8ce3-77ecb8f75745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:06.802829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f8eeb-beed-46f9-8ce3-77ecb8f75745 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.803108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f8eeb-beed-46f9-8ce3-77ecb8f75745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.803407 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f8eeb-beed-46f9-8ce3-77ecb8f75745 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.804076 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f8eeb-beed-46f9-8ce3-77ecb8f75745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:06.805683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f8eeb-beed-46f9-8ce3-77ecb8f75745 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:06.812784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1ee99-60de-4c4c-8377-4f5b1eaec502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:06.813331 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1ee99-60de-4c4c-8377-4f5b1eaec502 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.813611 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1ee99-60de-4c4c-8377-4f5b1eaec502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.813885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1ee99-60de-4c4c-8377-4f5b1eaec502 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.814535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1ee99-60de-4c4c-8377-4f5b1eaec502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:06.816186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1ee99-60de-4c4c-8377-4f5b1eaec502 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:06.816751 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 931/1841] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:06.820517 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f8eeb-beed-46f9-8ce3-77ecb8f75745 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:06.826779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 911/1842] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:06.830213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f8eeb-beed-46f9-8ce3-77ecb8f75745 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:06.830510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1ee99-60de-4c4c-8377-4f5b1eaec502 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:06.835265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a675e6b-aeb9-470d-8029-9e6eea266ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:06.835975 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a675e6b-aeb9-470d-8029-9e6eea266ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.836257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a675e6b-aeb9-470d-8029-9e6eea266ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.836542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a675e6b-aeb9-470d-8029-9e6eea266ec2 admin admin] PATH_INFO: `/v3/projects/012e36787dcf4daf966d29119dc8f4c7/users/3d4089961c324c7483382f64a943891d/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.838684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a675e6b-aeb9-470d-8029-9e6eea266ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=012e36787dcf4daf966d29119dc8f4c7, user_id=3d4089961c324c7483382f64a943891d, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:06.840930 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1ee99-60de-4c4c-8377-4f5b1eaec502 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:06.841450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a675e6b-aeb9-470d-8029-9e6eea266ec2 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:06.846644 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ae534-0402-461e-93b6-b5f3a7ce4695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:06.847475 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ae534-0402-461e-93b6-b5f3a7ce4695 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.847759 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ae534-0402-461e-93b6-b5f3a7ce4695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.848038 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ae534-0402-461e-93b6-b5f3a7ce4695 admin admin] PATH_INFO: `/v3/projects/20d1d40307a2438e95cac3a2b118a698/users/301a1e1105ea4c1b9b32336793df73a1/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.852281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 932/1843] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:06 2022] PUT /identity/v3/projects/012e36787dcf4daf966d29119dc8f4c7/users/3d4089961c324c7483382f64a943891d/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:06.867823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf89039-0fd1-4904-8ce0-db2c493c9c53 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.868293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf89039-0fd1-4904-8ce0-db2c493c9c53 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.868434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ae534-0402-461e-93b6-b5f3a7ce4695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20d1d40307a2438e95cac3a2b118a698, user_id=301a1e1105ea4c1b9b32336793df73a1, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:06.868674 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf89039-0fd1-4904-8ce0-db2c493c9c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.871393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ae534-0402-461e-93b6-b5f3a7ce4695 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:06.874001 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:06.888578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 912/1844] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:06 2022] PUT /identity/v3/projects/20d1d40307a2438e95cac3a2b118a698/users/301a1e1105ea4c1b9b32336793df73a1/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:06.899973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b2200-eb65-45da-9493-542b85aa7ccc None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:06.900312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b2200-eb65-45da-9493-542b85aa7ccc None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:06.901026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b2200-eb65-45da-9493-542b85aa7ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:06.901127 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-7bf89039-0fd1-4904-8ce0-db2c493c9c53 None None] MFA Rules not processed for user `3d4089961c324c7483382f64a943891d`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:06.901816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-7bf89039-0fd1-4904-8ce0-db2c493c9c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:06.927266 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-700b2200-eb65-45da-9493-542b85aa7ccc None None] MFA Rules not processed for user `301a1e1105ea4c1b9b32336793df73a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:06.928528 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-700b2200-eb65-45da-9493-542b85aa7ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:06.944164 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:06.974935 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 933/1845] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:06 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:06.987182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf89039-0fd1-4904-8ce0-db2c493c9c53 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:06.988632 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-7bf89039-0fd1-4904-8ce0-db2c493c9c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:06.994199 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 913/1846] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:07.005750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b2200-eb65-45da-9493-542b85aa7ccc None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:07.007125 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-700b2200-eb65-45da-9493-542b85aa7ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:07.048157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf89039-0fd1-4904-8ce0-db2c493c9c53 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:07.052316 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2128d-b96b-4849-90c4-f9ef184a9627 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:07.052981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2128d-b96b-4849-90c4-f9ef184a9627 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:07.053253 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2128d-b96b-4849-90c4-f9ef184a9627 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:07.053519 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2128d-b96b-4849-90c4-f9ef184a9627 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:07.063500 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a2128d-b96b-4849-90c4-f9ef184a9627 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:07.064960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a2128d-b96b-4849-90c4-f9ef184a9627 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:07.074644 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 934/1847] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:06 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:13:07.084823 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b2200-eb65-45da-9493-542b85aa7ccc None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:07.088703 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82858e6f-d379-41eb-ace8-9640c1a05b7a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57a5f04cb9674b86a6b4b9a472e1afc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:07.089344 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82858e6f-d379-41eb-ace8-9640c1a05b7a service glance] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:07.089620 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82858e6f-d379-41eb-ace8-9640c1a05b7a service glance] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:07.089891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82858e6f-d379-41eb-ace8-9640c1a05b7a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:07.098610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82858e6f-d379-41eb-ace8-9640c1a05b7a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:07.100088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82858e6f-d379-41eb-ace8-9640c1a05b7a service glance] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:07.110000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 914/1848] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:13:07.324235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2128d-b96b-4849-90c4-f9ef184a9627 service cinder] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:07.412637 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2128d-b96b-4849-90c4-f9ef184a9627 service cinder] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:07.418560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02ce4e-aa51-46b9-b814-be784e0d6fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:07.419358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02ce4e-aa51-46b9-b814-be784e0d6fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:07.419725 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02ce4e-aa51-46b9-b814-be784e0d6fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:07.420067 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02ce4e-aa51-46b9-b814-be784e0d6fad admin admin] PATH_INFO: `/v3/users/c38ef6b1f29f4e07ac607c64f28b0773` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:07.421281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02ce4e-aa51-46b9-b814-be784e0d6fad admin admin] RBAC: Authorizing `identity:delete_user(user_id=c38ef6b1f29f4e07ac607c64f28b0773)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:07.423309 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02ce4e-aa51-46b9-b814-be784e0d6fad admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:07.510306 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-ca02ce4e-aa51-46b9-b814-be784e0d6fad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c38ef6b1f29f4e07ac607c64f28b0773'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:07.511501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-ca02ce4e-aa51-46b9-b814-be784e0d6fad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c38ef6b1f29f4e07ac607c64f28b0773'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:07.512213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-ca02ce4e-aa51-46b9-b814-be784e0d6fad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c38ef6b1f29f4e07ac607c64f28b0773'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:07.522651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-ca02ce4e-aa51-46b9-b814-be784e0d6fad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c38ef6b1f29f4e07ac607c64f28b0773'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:07.544435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-ca02ce4e-aa51-46b9-b814-be784e0d6fad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c38ef6b1f29f4e07ac607c64f28b0773'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:07.558198 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:07.565677 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 935/1849] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:07 2022] DELETE /identity/v3/users/c38ef6b1f29f4e07ac607c64f28b0773 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:07.570316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f4831-e13b-47a8-a3c9-e36014927095 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:07.570570 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f4831-e13b-47a8-a3c9-e36014927095 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:07.570833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f4831-e13b-47a8-a3c9-e36014927095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:07.604896 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-d93f4831-e13b-47a8-a3c9-e36014927095 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:07.606223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d93f4831-e13b-47a8-a3c9-e36014927095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:07.877634 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 915/1850] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:07.888267 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02ce4e-aa51-46b9-b814-be784e0d6fad admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:07.889933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ca02ce4e-aa51-46b9-b814-be784e0d6fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:08.051503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02ce4e-aa51-46b9-b814-be784e0d6fad admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:08.057182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877af72-3457-4512-9182-afe701033bf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:08.057993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877af72-3457-4512-9182-afe701033bf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:08.058411 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877af72-3457-4512-9182-afe701033bf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:08.058854 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877af72-3457-4512-9182-afe701033bf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:08.079087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b877af72-3457-4512-9182-afe701033bf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:08.083443 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b877af72-3457-4512-9182-afe701033bf5 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:08.112464 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 936/1851] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:08.815558 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f4831-e13b-47a8-a3c9-e36014927095 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:08.817236 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d93f4831-e13b-47a8-a3c9-e36014927095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:08.831869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877af72-3457-4512-9182-afe701033bf5 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:08.834780 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b877af72-3457-4512-9182-afe701033bf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:08.847781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f4831-e13b-47a8-a3c9-e36014927095 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:08.855008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877af72-3457-4512-9182-afe701033bf5 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:08.858912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c45097-1e63-425a-a567-3750d96b3e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:08.859819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c45097-1e63-425a-a567-3750d96b3e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:08.860364 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c45097-1e63-425a-a567-3750d96b3e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:08.860639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c45097-1e63-425a-a567-3750d96b3e54 admin admin] PATH_INFO: `/v3/projects/2012102220bf443ca51a95e0a38c77d4` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:08.863560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c45097-1e63-425a-a567-3750d96b3e54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2012102220bf443ca51a95e0a38c77d4)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:08.866229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c45097-1e63-425a-a567-3750d96b3e54 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:08.869277 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37e857-4463-439b-a768-44c53832dbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:08.869277 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37e857-4463-439b-a768-44c53832dbe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:08.870517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:08.870642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37e857-4463-439b-a768-44c53832dbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:08.870950 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37e857-4463-439b-a768-44c53832dbe7 admin admin] PATH_INFO: `/v3/users/47d206b049a04052a2e40efcb071ef45` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:08.872356 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37e857-4463-439b-a768-44c53832dbe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47d206b049a04052a2e40efcb071ef45)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:08.884394 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37e857-4463-439b-a768-44c53832dbe7 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:08.946500 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-ae37e857-4463-439b-a768-44c53832dbe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47d206b049a04052a2e40efcb071ef45'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:08.947376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-ae37e857-4463-439b-a768-44c53832dbe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47d206b049a04052a2e40efcb071ef45'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:08.948251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-ae37e857-4463-439b-a768-44c53832dbe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47d206b049a04052a2e40efcb071ef45'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:08.953568 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-67c45097-1e63-425a-a567-3750d96b3e54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2012102220bf443ca51a95e0a38c77d4', 'request_id': 'req-67c45097-1e63-425a-a567-3750d96b3e54'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:08.957710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-ae37e857-4463-439b-a768-44c53832dbe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47d206b049a04052a2e40efcb071ef45'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:08.963197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-67c45097-1e63-425a-a567-3750d96b3e54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2012102220bf443ca51a95e0a38c77d4', 'request_id': 'req-67c45097-1e63-425a-a567-3750d96b3e54'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:08.973247 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-67c45097-1e63-425a-a567-3750d96b3e54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2012102220bf443ca51a95e0a38c77d4', 'request_id': 'req-67c45097-1e63-425a-a567-3750d96b3e54'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:08.979258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-ae37e857-4463-439b-a768-44c53832dbe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47d206b049a04052a2e40efcb071ef45'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:08.989195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-67c45097-1e63-425a-a567-3750d96b3e54 admin admin] The token cache is being invalidate because project 2012102220bf443ca51a95e0a38c77d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:08.989574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-67c45097-1e63-425a-a567-3750d96b3e54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:08.990124 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-67c45097-1e63-425a-a567-3750d96b3e54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:08.992155 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 916/1852] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:08 2022] DELETE /identity/v3/projects/2012102220bf443ca51a95e0a38c77d4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:08.996476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c45097-1e63-425a-a567-3750d96b3e54 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:08.996638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 937/1853] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:08 2022] DELETE /identity/v3/users/47d206b049a04052a2e40efcb071ef45 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:08.999356 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-67c45097-1e63-425a-a567-3750d96b3e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:09.001745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca4db5-eb8c-4fa1-b3ca-6c5cd6d564d1 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:09.002082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca4db5-eb8c-4fa1-b3ca-6c5cd6d564d1 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:09.002301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca4db5-eb8c-4fa1-b3ca-6c5cd6d564d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:09.031211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-b9ca4db5-eb8c-4fa1-b3ca-6c5cd6d564d1 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:09.032519 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b9ca4db5-eb8c-4fa1-b3ca-6c5cd6d564d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:09.129764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c45097-1e63-425a-a567-3750d96b3e54 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:09.134083 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 938/1854] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:09.146337 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca4db5-eb8c-4fa1-b3ca-6c5cd6d564d1 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:09.149211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b9ca4db5-eb8c-4fa1-b3ca-6c5cd6d564d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:09.151718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778009d-648b-4643-8650-bfb5a400376b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:09.152859 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778009d-648b-4643-8650-bfb5a400376b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:09.158074 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778009d-648b-4643-8650-bfb5a400376b admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:09.158621 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778009d-648b-4643-8650-bfb5a400376b admin admin] PATH_INFO: `/v3/users/3d4089961c324c7483382f64a943891d` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:09.160305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778009d-648b-4643-8650-bfb5a400376b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d4089961c324c7483382f64a943891d)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:09.162893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778009d-648b-4643-8650-bfb5a400376b admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:09.177197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:09.244285 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca4db5-eb8c-4fa1-b3ca-6c5cd6d564d1 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:09.247593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f97ac-5452-4bf1-b94f-b9420515c1d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:09.248150 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f97ac-5452-4bf1-b94f-b9420515c1d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:09.248429 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f97ac-5452-4bf1-b94f-b9420515c1d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:09.248699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f97ac-5452-4bf1-b94f-b9420515c1d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:09.262488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f97ac-5452-4bf1-b94f-b9420515c1d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:09.264034 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f97ac-5452-4bf1-b94f-b9420515c1d3 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:09.265024 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-3778009d-648b-4643-8650-bfb5a400376b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d4089961c324c7483382f64a943891d'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:09.266223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-3778009d-648b-4643-8650-bfb5a400376b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d4089961c324c7483382f64a943891d'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:09.266937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-3778009d-648b-4643-8650-bfb5a400376b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d4089961c324c7483382f64a943891d'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:09.279676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 939/1855] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:09.285557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-3778009d-648b-4643-8650-bfb5a400376b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d4089961c324c7483382f64a943891d'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:09.303077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-3778009d-648b-4643-8650-bfb5a400376b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d4089961c324c7483382f64a943891d'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:09.322437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 917/1856] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:08 2022] DELETE /identity/v3/users/3d4089961c324c7483382f64a943891d => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:09.902983 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f97ac-5452-4bf1-b94f-b9420515c1d3 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:09.905774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d93f97ac-5452-4bf1-b94f-b9420515c1d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:09.949105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f97ac-5452-4bf1-b94f-b9420515c1d3 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:10.106913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45137f5c-9d8b-47f3-84fe-1f73f6d55097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:10.107862 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45137f5c-9d8b-47f3-84fe-1f73f6d55097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:10.108866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45137f5c-9d8b-47f3-84fe-1f73f6d55097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:10.109247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45137f5c-9d8b-47f3-84fe-1f73f6d55097 admin admin] PATH_INFO: `/v3/projects/802af76392bd4ad2b20866c0f6197ca2` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:10.111379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45137f5c-9d8b-47f3-84fe-1f73f6d55097 admin admin] RBAC: Authorizing `identity:delete_project(project_id=802af76392bd4ad2b20866c0f6197ca2)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:10.114213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45137f5c-9d8b-47f3-84fe-1f73f6d55097 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:10.177646 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778009d-648b-4643-8650-bfb5a400376b admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:10.179325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3778009d-648b-4643-8650-bfb5a400376b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:10.195208 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778009d-648b-4643-8650-bfb5a400376b admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:10.201212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:10.208326 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-45137f5c-9d8b-47f3-84fe-1f73f6d55097 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '802af76392bd4ad2b20866c0f6197ca2', 'request_id': 'req-45137f5c-9d8b-47f3-84fe-1f73f6d55097'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.226498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-45137f5c-9d8b-47f3-84fe-1f73f6d55097 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '802af76392bd4ad2b20866c0f6197ca2', 'request_id': 'req-45137f5c-9d8b-47f3-84fe-1f73f6d55097'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.242756 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-45137f5c-9d8b-47f3-84fe-1f73f6d55097 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '802af76392bd4ad2b20866c0f6197ca2', 'request_id': 'req-45137f5c-9d8b-47f3-84fe-1f73f6d55097'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.261260 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-45137f5c-9d8b-47f3-84fe-1f73f6d55097 admin admin] The token cache is being invalidate because project 802af76392bd4ad2b20866c0f6197ca2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:10.261555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-45137f5c-9d8b-47f3-84fe-1f73f6d55097 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.263367 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-45137f5c-9d8b-47f3-84fe-1f73f6d55097 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.268778 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 940/1857] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:09 2022] DELETE /identity/v3/projects/802af76392bd4ad2b20866c0f6197ca2 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:10.281849 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45137f5c-9d8b-47f3-84fe-1f73f6d55097 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:10.284410 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-45137f5c-9d8b-47f3-84fe-1f73f6d55097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:10.296781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926dd22-25b1-49b5-b556-90e358549026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:10.297785 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926dd22-25b1-49b5-b556-90e358549026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:10.298173 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926dd22-25b1-49b5-b556-90e358549026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:10.298553 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926dd22-25b1-49b5-b556-90e358549026 admin admin] PATH_INFO: `/v3/projects/012e36787dcf4daf966d29119dc8f4c7` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:10.300054 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926dd22-25b1-49b5-b556-90e358549026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=012e36787dcf4daf966d29119dc8f4c7)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:10.302218 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926dd22-25b1-49b5-b556-90e358549026 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:10.317742 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45137f5c-9d8b-47f3-84fe-1f73f6d55097 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:10.325265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26249565-647b-4e16-b0d7-68a6614bdb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:10.325832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26249565-647b-4e16-b0d7-68a6614bdb59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:10.326218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26249565-647b-4e16-b0d7-68a6614bdb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:10.326761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26249565-647b-4e16-b0d7-68a6614bdb59 admin admin] PATH_INFO: `/v3/users/7798edef75a046b784e6782291717843` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:10.328730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26249565-647b-4e16-b0d7-68a6614bdb59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7798edef75a046b784e6782291717843)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:10.330402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26249565-647b-4e16-b0d7-68a6614bdb59 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:10.378465 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1926dd22-25b1-49b5-b556-90e358549026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '012e36787dcf4daf966d29119dc8f4c7', 'request_id': 'req-1926dd22-25b1-49b5-b556-90e358549026'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.388362 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1926dd22-25b1-49b5-b556-90e358549026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '012e36787dcf4daf966d29119dc8f4c7', 'request_id': 'req-1926dd22-25b1-49b5-b556-90e358549026'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.394665 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:10.399578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1926dd22-25b1-49b5-b556-90e358549026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '012e36787dcf4daf966d29119dc8f4c7', 'request_id': 'req-1926dd22-25b1-49b5-b556-90e358549026'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.405114 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-26249565-647b-4e16-b0d7-68a6614bdb59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7798edef75a046b784e6782291717843'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.406492 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-26249565-647b-4e16-b0d7-68a6614bdb59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7798edef75a046b784e6782291717843'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.408557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-26249565-647b-4e16-b0d7-68a6614bdb59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7798edef75a046b784e6782291717843'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.416428 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1926dd22-25b1-49b5-b556-90e358549026 admin admin] The token cache is being invalidate because project 012e36787dcf4daf966d29119dc8f4c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:10.416725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1926dd22-25b1-49b5-b556-90e358549026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.417389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1926dd22-25b1-49b5-b556-90e358549026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.419312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 918/1858] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:10 2022] DELETE /identity/v3/projects/012e36787dcf4daf966d29119dc8f4c7 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:10.421646 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-26249565-647b-4e16-b0d7-68a6614bdb59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7798edef75a046b784e6782291717843'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.423768 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f909f5-a6d1-4d7b-9f73-616d265babb8 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:10.424053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f909f5-a6d1-4d7b-9f73-616d265babb8 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:10.424324 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f909f5-a6d1-4d7b-9f73-616d265babb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:10.435718 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-26249565-647b-4e16-b0d7-68a6614bdb59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7798edef75a046b784e6782291717843'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.444607 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-83f909f5-a6d1-4d7b-9f73-616d265babb8 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:10.445870 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-83f909f5-a6d1-4d7b-9f73-616d265babb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:10.457268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 941/1859] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:10 2022] DELETE /identity/v3/users/7798edef75a046b784e6782291717843 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:10.532468 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 919/1860] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:10.540757 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4983efc-6727-4821-90ef-26000afd0ae5 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:10.541115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4983efc-6727-4821-90ef-26000afd0ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:10.541459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4983efc-6727-4821-90ef-26000afd0ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:10.561721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-e4983efc-6727-4821-90ef-26000afd0ae5 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:10.563001 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e4983efc-6727-4821-90ef-26000afd0ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:10.626624 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 942/1861] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:10.630989 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f909f5-a6d1-4d7b-9f73-616d265babb8 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:10.641068 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f909f5-a6d1-4d7b-9f73-616d265babb8 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:10.645702 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e2cba-19be-4c22-a5cd-5be393580415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:10.646312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e2cba-19be-4c22-a5cd-5be393580415 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:10.646593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e2cba-19be-4c22-a5cd-5be393580415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:10.646897 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e2cba-19be-4c22-a5cd-5be393580415 admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:10.647585 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e2cba-19be-4c22-a5cd-5be393580415 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:10.647935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e2cba-19be-4c22-a5cd-5be393580415 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:10.649679 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e2cba-19be-4c22-a5cd-5be393580415 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:10.666104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 920/1862] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:13:10.670505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4983efc-6727-4821-90ef-26000afd0ae5 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:10.679845 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4983efc-6727-4821-90ef-26000afd0ae5 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:10.690510 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477b6f5-5c26-433d-8135-8dff6429f4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:10.690814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477b6f5-5c26-433d-8135-8dff6429f4bc admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:10.691191 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477b6f5-5c26-433d-8135-8dff6429f4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:10.691616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477b6f5-5c26-433d-8135-8dff6429f4bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:10.692265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477b6f5-5c26-433d-8135-8dff6429f4bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:10.694515 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477b6f5-5c26-433d-8135-8dff6429f4bc admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:10.725467 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 943/1863] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:10 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:10.729812 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e2cba-19be-4c22-a5cd-5be393580415 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:10.736082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477b6f5-5c26-433d-8135-8dff6429f4bc admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:10.737852 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e477b6f5-5c26-433d-8135-8dff6429f4bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:10.740367 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e2cba-19be-4c22-a5cd-5be393580415 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:10.768364 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:10.805371 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fe4eb-402e-4c1d-8c1b-b1d4aaabe402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:10.806315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fe4eb-402e-4c1d-8c1b-b1d4aaabe402 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:10.806485 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fe4eb-402e-4c1d-8c1b-b1d4aaabe402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:10.806692 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fe4eb-402e-4c1d-8c1b-b1d4aaabe402 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:10.807335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080fe4eb-402e-4c1d-8c1b-b1d4aaabe402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:10.809304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080fe4eb-402e-4c1d-8c1b-b1d4aaabe402 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:10.814605 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477b6f5-5c26-433d-8135-8dff6429f4bc admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:10.819659 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d506cf-a39e-4b22-9240-bae9e3897c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:10.819986 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d506cf-a39e-4b22-9240-bae9e3897c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:10.820287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d506cf-a39e-4b22-9240-bae9e3897c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:10.820569 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d506cf-a39e-4b22-9240-bae9e3897c51 admin admin] PATH_INFO: `/v3/projects/ea388c62925345fba53df92ee1201c69` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:10.821871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d506cf-a39e-4b22-9240-bae9e3897c51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea388c62925345fba53df92ee1201c69)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:10.824504 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d506cf-a39e-4b22-9240-bae9e3897c51 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:10.832161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 921/1864] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:10 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:10.836324 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fe4eb-402e-4c1d-8c1b-b1d4aaabe402 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:10.846397 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fe4eb-402e-4c1d-8c1b-b1d4aaabe402 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:10.851094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904a3b4-a2a0-410a-bad3-240c9d298823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:10.851775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904a3b4-a2a0-410a-bad3-240c9d298823 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:10.852099 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904a3b4-a2a0-410a-bad3-240c9d298823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:10.852413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904a3b4-a2a0-410a-bad3-240c9d298823 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:10.853232 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904a3b4-a2a0-410a-bad3-240c9d298823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:10.855037 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904a3b4-a2a0-410a-bad3-240c9d298823 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:10.866076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 922/1865] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:10.870054 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904a3b4-a2a0-410a-bad3-240c9d298823 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:10.870828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:10.879966 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-32d506cf-a39e-4b22-9240-bae9e3897c51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea388c62925345fba53df92ee1201c69', 'request_id': 'req-32d506cf-a39e-4b22-9240-bae9e3897c51'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.881475 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904a3b4-a2a0-410a-bad3-240c9d298823 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:10.889364 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-32d506cf-a39e-4b22-9240-bae9e3897c51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea388c62925345fba53df92ee1201c69', 'request_id': 'req-32d506cf-a39e-4b22-9240-bae9e3897c51'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.899042 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-32d506cf-a39e-4b22-9240-bae9e3897c51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea388c62925345fba53df92ee1201c69', 'request_id': 'req-32d506cf-a39e-4b22-9240-bae9e3897c51'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.911043 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-32d506cf-a39e-4b22-9240-bae9e3897c51 admin admin] The token cache is being invalidate because project ea388c62925345fba53df92ee1201c69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:10.911463 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-32d506cf-a39e-4b22-9240-bae9e3897c51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.912128 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-32d506cf-a39e-4b22-9240-bae9e3897c51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:10.913851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 944/1866] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:10 2022] DELETE /identity/v3/projects/ea388c62925345fba53df92ee1201c69 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:10.918459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff6f0b-ed1c-4520-9abf-91de185dd007 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:10.918740 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff6f0b-ed1c-4520-9abf-91de185dd007 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:10.919013 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff6f0b-ed1c-4520-9abf-91de185dd007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:10.939680 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-02ff6f0b-ed1c-4520-9abf-91de185dd007 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:10.940785 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554c0af-e4f9-4bce-8ee0-1a88663bf247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:10.940915 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-02ff6f0b-ed1c-4520-9abf-91de185dd007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:10.941400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554c0af-e4f9-4bce-8ee0-1a88663bf247 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:10.941678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554c0af-e4f9-4bce-8ee0-1a88663bf247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:10.941951 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554c0af-e4f9-4bce-8ee0-1a88663bf247 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:10.942633 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554c0af-e4f9-4bce-8ee0-1a88663bf247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:10.944281 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554c0af-e4f9-4bce-8ee0-1a88663bf247 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:10.948818 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 945/1867] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:10.954121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 923/1868] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:10.956659 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff26e6-048f-4fab-9b91-36c45a0366ba None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:10.957105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff26e6-048f-4fab-9b91-36c45a0366ba None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:10.957278 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff26e6-048f-4fab-9b91-36c45a0366ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:10.958361 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554c0af-e4f9-4bce-8ee0-1a88663bf247 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:10.960140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9554c0af-e4f9-4bce-8ee0-1a88663bf247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:10.972455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554c0af-e4f9-4bce-8ee0-1a88663bf247 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:10.977031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-81ff26e6-048f-4fab-9b91-36c45a0366ba None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:10.977308 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc2377-05d6-40b2-b14e-0358c5f9b11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:10.978009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc2377-05d6-40b2-b14e-0358c5f9b11b admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:10.978295 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc2377-05d6-40b2-b14e-0358c5f9b11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:10.978418 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-81ff26e6-048f-4fab-9b91-36c45a0366ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:10.978702 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc2377-05d6-40b2-b14e-0358c5f9b11b admin admin] PATH_INFO: `/v3/projects/da39cfc285944558a9b45ac8deb96f12/users/e206598ef86043ffbe5e63c8b5d80cd8/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:10.987011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 946/1869] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:10.991664 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff26e6-048f-4fab-9b91-36c45a0366ba None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:10.994563 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc2377-05d6-40b2-b14e-0358c5f9b11b admin admin] RBAC: Authorizing `identity:create_grant(project_id=da39cfc285944558a9b45ac8deb96f12, user_id=e206598ef86043ffbe5e63c8b5d80cd8, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:10.997428 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc2377-05d6-40b2-b14e-0358c5f9b11b admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.001006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff26e6-048f-4fab-9b91-36c45a0366ba None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:11.006114 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b62bd-3211-47e7-9d77-c73b965cab08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:11.006752 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b62bd-3211-47e7-9d77-c73b965cab08 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.007042 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b62bd-3211-47e7-9d77-c73b965cab08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.007287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b62bd-3211-47e7-9d77-c73b965cab08 admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.007959 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b62bd-3211-47e7-9d77-c73b965cab08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:11.008300 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b62bd-3211-47e7-9d77-c73b965cab08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:11.010005 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b62bd-3211-47e7-9d77-c73b965cab08 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.012120 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 924/1870] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:10 2022] PUT /identity/v3/projects/da39cfc285944558a9b45ac8deb96f12/users/e206598ef86043ffbe5e63c8b5d80cd8/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:11.022332 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 947/1871] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:13:11.023311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d264f-1fd8-4aa9-8ad6-93deb2895613 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.023631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d264f-1fd8-4aa9-8ad6-93deb2895613 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.023935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d264f-1fd8-4aa9-8ad6-93deb2895613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.026635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b62bd-3211-47e7-9d77-c73b965cab08 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:11.037226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b62bd-3211-47e7-9d77-c73b965cab08 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:11.051804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-3e0d264f-1fd8-4aa9-8ad6-93deb2895613 None None] MFA Rules not processed for user `e206598ef86043ffbe5e63c8b5d80cd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:11.053006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3e0d264f-1fd8-4aa9-8ad6-93deb2895613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:11.119998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d65e0b-fef6-4c3c-abd8-21bf6f2c3570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:11.120627 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d65e0b-fef6-4c3c-abd8-21bf6f2c3570 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.120907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d65e0b-fef6-4c3c-abd8-21bf6f2c3570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.121174 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d65e0b-fef6-4c3c-abd8-21bf6f2c3570 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.121963 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d65e0b-fef6-4c3c-abd8-21bf6f2c3570 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:11.123974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d65e0b-fef6-4c3c-abd8-21bf6f2c3570 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.135480 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 925/1872] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:11.140107 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d264f-1fd8-4aa9-8ad6-93deb2895613 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:11.151757 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d264f-1fd8-4aa9-8ad6-93deb2895613 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:11.157116 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 948/1873] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:11 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:11.157262 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:11.158169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d65e0b-fef6-4c3c-abd8-21bf6f2c3570 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:11.213532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb1a89-611b-4b0c-8855-d2c23ca4df3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:11.214157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb1a89-611b-4b0c-8855-d2c23ca4df3f admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.214429 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb1a89-611b-4b0c-8855-d2c23ca4df3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.214704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb1a89-611b-4b0c-8855-d2c23ca4df3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.215342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb1a89-611b-4b0c-8855-d2c23ca4df3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:11.217319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb1a89-611b-4b0c-8855-d2c23ca4df3f admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.224969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d65e0b-fef6-4c3c-abd8-21bf6f2c3570 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:11.230479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8442b99-0136-4788-9404-63dd9673f0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:11.231097 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8442b99-0136-4788-9404-63dd9673f0ee admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.231374 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8442b99-0136-4788-9404-63dd9673f0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.231679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8442b99-0136-4788-9404-63dd9673f0ee admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.232331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8442b99-0136-4788-9404-63dd9673f0ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:11.234371 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8442b99-0136-4788-9404-63dd9673f0ee admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.248604 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 926/1874] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:11 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:11.252884 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb1a89-611b-4b0c-8855-d2c23ca4df3f admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:11.259331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 949/1875] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:11 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:11.263664 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8442b99-0136-4788-9404-63dd9673f0ee admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:11.274263 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8442b99-0136-4788-9404-63dd9673f0ee admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:11.288318 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:11.335126 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb1a89-611b-4b0c-8855-d2c23ca4df3f admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:11.340013 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b487b5-ca4b-4290-b075-db542c23df5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:11.340638 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b487b5-ca4b-4290-b075-db542c23df5e admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.340921 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b487b5-ca4b-4290-b075-db542c23df5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.341193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b487b5-ca4b-4290-b075-db542c23df5e admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.341824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b487b5-ca4b-4290-b075-db542c23df5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:11.355501 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b487b5-ca4b-4290-b075-db542c23df5e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.355636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501cef2c-af45-4958-a780-cd2ad8ce363d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:11.355636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501cef2c-af45-4958-a780-cd2ad8ce363d admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.355636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501cef2c-af45-4958-a780-cd2ad8ce363d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.355636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501cef2c-af45-4958-a780-cd2ad8ce363d admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.355985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501cef2c-af45-4958-a780-cd2ad8ce363d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:11.355985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501cef2c-af45-4958-a780-cd2ad8ce363d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.363441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 950/1876] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:11.366488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501cef2c-af45-4958-a780-cd2ad8ce363d admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:11.368824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 927/1877] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:11 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:11.372483 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b487b5-ca4b-4290-b075-db542c23df5e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:11.377380 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501cef2c-af45-4958-a780-cd2ad8ce363d admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:11.382298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce076e7-da96-4308-8f71-0ea03efd25bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:11.382814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b487b5-ca4b-4290-b075-db542c23df5e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:11.382928 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce076e7-da96-4308-8f71-0ea03efd25bf admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.383221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce076e7-da96-4308-8f71-0ea03efd25bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.383604 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce076e7-da96-4308-8f71-0ea03efd25bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.384180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce076e7-da96-4308-8f71-0ea03efd25bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:11.386117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce076e7-da96-4308-8f71-0ea03efd25bf admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.387718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097d146-2f34-4746-8b8b-94ba2510941d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:11.388381 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097d146-2f34-4746-8b8b-94ba2510941d admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.388686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097d146-2f34-4746-8b8b-94ba2510941d admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.388958 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097d146-2f34-4746-8b8b-94ba2510941d admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.389632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097d146-2f34-4746-8b8b-94ba2510941d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:11.391223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097d146-2f34-4746-8b8b-94ba2510941d admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.396599 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 951/1878] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:11.401007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce076e7-da96-4308-8f71-0ea03efd25bf admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:11.402641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 928/1879] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:11.406531 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097d146-2f34-4746-8b8b-94ba2510941d admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:11.411298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce076e7-da96-4308-8f71-0ea03efd25bf admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:11.416306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097d146-2f34-4746-8b8b-94ba2510941d admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:11.416679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00afbb6-6f1a-46c4-9c7b-3f86bc9b014a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:11.417332 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00afbb6-6f1a-46c4-9c7b-3f86bc9b014a admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.417615 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00afbb6-6f1a-46c4-9c7b-3f86bc9b014a admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.417890 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00afbb6-6f1a-46c4-9c7b-3f86bc9b014a admin admin] PATH_INFO: `/v3/projects/ab05ecba3c9449bbb2071623d8fc4881/users/41a5bcad913143e59b114b163cb3f46c/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.421082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42df5c-3fa7-4685-940a-8c1216b3f35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:11.421669 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42df5c-3fa7-4685-940a-8c1216b3f35d admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.421949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42df5c-3fa7-4685-940a-8c1216b3f35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.422219 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42df5c-3fa7-4685-940a-8c1216b3f35d admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.422853 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42df5c-3fa7-4685-940a-8c1216b3f35d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:11.424502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42df5c-3fa7-4685-940a-8c1216b3f35d admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.434615 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00afbb6-6f1a-46c4-9c7b-3f86bc9b014a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab05ecba3c9449bbb2071623d8fc4881, user_id=41a5bcad913143e59b114b163cb3f46c, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:11.434776 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 929/1880] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:11.437479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00afbb6-6f1a-46c4-9c7b-3f86bc9b014a admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.438758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42df5c-3fa7-4685-940a-8c1216b3f35d admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:11.448777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42df5c-3fa7-4685-940a-8c1216b3f35d admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:11.452171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 952/1881] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:11 2022] PUT /identity/v3/projects/ab05ecba3c9449bbb2071623d8fc4881/users/41a5bcad913143e59b114b163cb3f46c/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:11.463112 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ea022-f319-4e41-8486-6a61eb823ab4 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.463397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ea022-f319-4e41-8486-6a61eb823ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.463738 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ea022-f319-4e41-8486-6a61eb823ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.492685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-c39ea022-f319-4e41-8486-6a61eb823ab4 None None] MFA Rules not processed for user `41a5bcad913143e59b114b163cb3f46c`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:11.494051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c39ea022-f319-4e41-8486-6a61eb823ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:11.521141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5b02d-2706-40de-bcf0-e40c4eab9fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:11.521899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5b02d-2706-40de-bcf0-e40c4eab9fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.522214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5b02d-2706-40de-bcf0-e40c4eab9fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.523003 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5b02d-2706-40de-bcf0-e40c4eab9fd0 admin admin] PATH_INFO: `/v3/projects/d9ed83b36f0d4cd5b158c197aa2b10a8/users/4a91f43c74dc48ada0b8dabf281c5d0f/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.525799 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:11.552351 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5b02d-2706-40de-bcf0-e40c4eab9fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ed83b36f0d4cd5b158c197aa2b10a8, user_id=4a91f43c74dc48ada0b8dabf281c5d0f, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:11.556458 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5b02d-2706-40de-bcf0-e40c4eab9fd0 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.574713 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 930/1882] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:11 2022] PUT /identity/v3/projects/d9ed83b36f0d4cd5b158c197aa2b10a8/users/4a91f43c74dc48ada0b8dabf281c5d0f/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:11.586262 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 953/1883] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:11.586550 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc538eb5-0ddf-448b-a773-23b279764e2c None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.586848 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc538eb5-0ddf-448b-a773-23b279764e2c None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.587134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc538eb5-0ddf-448b-a773-23b279764e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.590926 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ea022-f319-4e41-8486-6a61eb823ab4 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:11.602209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ea022-f319-4e41-8486-6a61eb823ab4 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:11.620375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-cc538eb5-0ddf-448b-a773-23b279764e2c None None] MFA Rules not processed for user `4a91f43c74dc48ada0b8dabf281c5d0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:11.621966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-cc538eb5-0ddf-448b-a773-23b279764e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:11.668233 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff47ad9-5285-42ea-abb2-23caff8fab31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:11.668719 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff47ad9-5285-42ea-abb2-23caff8fab31 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.669003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff47ad9-5285-42ea-abb2-23caff8fab31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.669277 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff47ad9-5285-42ea-abb2-23caff8fab31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.669907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff47ad9-5285-42ea-abb2-23caff8fab31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:11.672902 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff47ad9-5285-42ea-abb2-23caff8fab31 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.687895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:11.700023 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 931/1884] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:11.703925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc538eb5-0ddf-448b-a773-23b279764e2c None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:11.712415 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 954/1885] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:11 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:11.713381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff47ad9-5285-42ea-abb2-23caff8fab31 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:11.720913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc538eb5-0ddf-448b-a773-23b279764e2c None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:11.786807 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff47ad9-5285-42ea-abb2-23caff8fab31 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:11.791658 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf209b-abf6-443f-9f9d-e8eac8c6137a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:11.792282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf209b-abf6-443f-9f9d-e8eac8c6137a admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.792578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf209b-abf6-443f-9f9d-e8eac8c6137a admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.792847 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf209b-abf6-443f-9f9d-e8eac8c6137a admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.793473 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf209b-abf6-443f-9f9d-e8eac8c6137a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:11.795699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf209b-abf6-443f-9f9d-e8eac8c6137a admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.806430 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca36896-9bfa-4b07-8bfe-63fa64f3063e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:11.807185 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca36896-9bfa-4b07-8bfe-63fa64f3063e admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.807466 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca36896-9bfa-4b07-8bfe-63fa64f3063e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.807769 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca36896-9bfa-4b07-8bfe-63fa64f3063e admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.808397 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca36896-9bfa-4b07-8bfe-63fa64f3063e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:11.810544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca36896-9bfa-4b07-8bfe-63fa64f3063e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.821032 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 955/1886] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:11 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:11.822342 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf209b-abf6-443f-9f9d-e8eac8c6137a admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:11.833024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf209b-abf6-443f-9f9d-e8eac8c6137a admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:11.837913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7250e2-d03b-48aa-ac0c-2e6e939e044a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:11.838503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7250e2-d03b-48aa-ac0c-2e6e939e044a admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.838778 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7250e2-d03b-48aa-ac0c-2e6e939e044a admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.839046 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7250e2-d03b-48aa-ac0c-2e6e939e044a admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.839708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7250e2-d03b-48aa-ac0c-2e6e939e044a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:11.840584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 932/1887] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:11 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:11.841451 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7250e2-d03b-48aa-ac0c-2e6e939e044a admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.844997 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca36896-9bfa-4b07-8bfe-63fa64f3063e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:11.857388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 956/1888] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:11.859015 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7250e2-d03b-48aa-ac0c-2e6e939e044a admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:11.869942 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7250e2-d03b-48aa-ac0c-2e6e939e044a admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:11.908124 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:11.912379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca36896-9bfa-4b07-8bfe-63fa64f3063e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:11.922083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bebba97-8fa8-4966-831b-86c7622173af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:11.922732 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bebba97-8fa8-4966-831b-86c7622173af admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.923329 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bebba97-8fa8-4966-831b-86c7622173af admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.924145 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bebba97-8fa8-4966-831b-86c7622173af admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.924820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bebba97-8fa8-4966-831b-86c7622173af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:11.927094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bebba97-8fa8-4966-831b-86c7622173af admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.939972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8ff4b-4b3b-4372-b5e5-cafb974fc813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:11.940721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8ff4b-4b3b-4372-b5e5-cafb974fc813 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.941126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8ff4b-4b3b-4372-b5e5-cafb974fc813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.941355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8ff4b-4b3b-4372-b5e5-cafb974fc813 admin admin] PATH_INFO: `/v3/projects/55a46ed7e8a34322995108bf5c16f5fa/users/7237e807baea4698a54c3f938dd8629a/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.955369 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 933/1889] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:11 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:11.956910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bebba97-8fa8-4966-831b-86c7622173af admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:11.961421 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8ff4b-4b3b-4372-b5e5-cafb974fc813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55a46ed7e8a34322995108bf5c16f5fa, user_id=7237e807baea4698a54c3f938dd8629a, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:11.964710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8ff4b-4b3b-4372-b5e5-cafb974fc813 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.967247 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bebba97-8fa8-4966-831b-86c7622173af admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:11.972205 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923305ec-f583-46c8-a7eb-9aa8f3cd1e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:11.972772 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923305ec-f583-46c8-a7eb-9aa8f3cd1e1f admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:11.973044 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923305ec-f583-46c8-a7eb-9aa8f3cd1e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:11.973400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923305ec-f583-46c8-a7eb-9aa8f3cd1e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:11.974126 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923305ec-f583-46c8-a7eb-9aa8f3cd1e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:11.977024 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923305ec-f583-46c8-a7eb-9aa8f3cd1e1f admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:11.981061 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 957/1890] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:11 2022] PUT /identity/v3/projects/55a46ed7e8a34322995108bf5c16f5fa/users/7237e807baea4698a54c3f938dd8629a/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:11.985391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8ff4b-4b3b-4372-b5e5-cafb974fc813 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:11.990862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 934/1891] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:11.992673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923305ec-f583-46c8-a7eb-9aa8f3cd1e1f admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:12.004564 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923305ec-f583-46c8-a7eb-9aa8f3cd1e1f admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:12.038819 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:12.053517 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8ff4b-4b3b-4372-b5e5-cafb974fc813 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:12.058320 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ddd7f-c4ed-4bdb-b0da-f4196f60ef08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:12.058932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ddd7f-c4ed-4bdb-b0da-f4196f60ef08 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:12.059207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ddd7f-c4ed-4bdb-b0da-f4196f60ef08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:12.059525 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ddd7f-c4ed-4bdb-b0da-f4196f60ef08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:12.060177 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ddd7f-c4ed-4bdb-b0da-f4196f60ef08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:12.061782 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ddd7f-c4ed-4bdb-b0da-f4196f60ef08 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:12.063455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d687bcb-6c24-41b6-8171-ddea6ad1805c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:12.063669 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d687bcb-6c24-41b6-8171-ddea6ad1805c admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:12.063955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d687bcb-6c24-41b6-8171-ddea6ad1805c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:12.064237 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d687bcb-6c24-41b6-8171-ddea6ad1805c admin admin] PATH_INFO: `/v3/projects/929d0e35319f45babbf281bfbfb2c66f/users/067f97a0734d428c99f3d041c55f5a6a/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:12.072705 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 958/1892] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:12.076969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ddd7f-c4ed-4bdb-b0da-f4196f60ef08 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:12.081103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d687bcb-6c24-41b6-8171-ddea6ad1805c admin admin] RBAC: Authorizing `identity:create_grant(project_id=929d0e35319f45babbf281bfbfb2c66f, user_id=067f97a0734d428c99f3d041c55f5a6a, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:12.084148 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d687bcb-6c24-41b6-8171-ddea6ad1805c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:12.088814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ddd7f-c4ed-4bdb-b0da-f4196f60ef08 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:12.094417 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152e86c-2847-476a-b210-1a25f1a3f54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:12.095255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152e86c-2847-476a-b210-1a25f1a3f54b admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:12.095550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152e86c-2847-476a-b210-1a25f1a3f54b admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:12.095861 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152e86c-2847-476a-b210-1a25f1a3f54b admin admin] PATH_INFO: `/v3/projects/55a46ed7e8a34322995108bf5c16f5fa/users/7237e807baea4698a54c3f938dd8629a/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:12.098443 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152e86c-2847-476a-b210-1a25f1a3f54b admin admin] RBAC: Authorizing `identity:create_grant(project_id=55a46ed7e8a34322995108bf5c16f5fa, user_id=7237e807baea4698a54c3f938dd8629a, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:12.102514 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152e86c-2847-476a-b210-1a25f1a3f54b admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:12.103186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 935/1893] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:11 2022] PUT /identity/v3/projects/929d0e35319f45babbf281bfbfb2c66f/users/067f97a0734d428c99f3d041c55f5a6a/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:12.107137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d687bcb-6c24-41b6-8171-ddea6ad1805c admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:12.120414 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 959/1894] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:12 2022] PUT /identity/v3/projects/55a46ed7e8a34322995108bf5c16f5fa/users/7237e807baea4698a54c3f938dd8629a/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:12.131910 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f3620-9f6f-40b3-9152-4ebbc516df6c None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:12.132267 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f3620-9f6f-40b3-9152-4ebbc516df6c None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:12.132547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f3620-9f6f-40b3-9152-4ebbc516df6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:12.166395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-616f3620-9f6f-40b3-9152-4ebbc516df6c None None] MFA Rules not processed for user `7237e807baea4698a54c3f938dd8629a`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:12.167655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-616f3620-9f6f-40b3-9152-4ebbc516df6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:12.189428 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d687bcb-6c24-41b6-8171-ddea6ad1805c admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:12.243134 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 960/1895] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:12.253197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae558edb-93f9-40d7-9804-ac8325c66626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:12.253881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae558edb-93f9-40d7-9804-ac8325c66626 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:12.254163 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae558edb-93f9-40d7-9804-ac8325c66626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:12.254441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae558edb-93f9-40d7-9804-ac8325c66626 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:12.256224 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f3620-9f6f-40b3-9152-4ebbc516df6c None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:12.256959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae558edb-93f9-40d7-9804-ac8325c66626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:12.257721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-616f3620-9f6f-40b3-9152-4ebbc516df6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:12.258693 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae558edb-93f9-40d7-9804-ac8325c66626 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:12.272519 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 936/1896] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:12.273499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae558edb-93f9-40d7-9804-ac8325c66626 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:12.278622 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:12.284440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae558edb-93f9-40d7-9804-ac8325c66626 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:12.290808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab357ad-fbce-4239-890b-22943bea0358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:12.291563 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab357ad-fbce-4239-890b-22943bea0358 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:12.291909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab357ad-fbce-4239-890b-22943bea0358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:12.292253 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab357ad-fbce-4239-890b-22943bea0358 admin admin] PATH_INFO: `/v3/projects/929d0e35319f45babbf281bfbfb2c66f/users/067f97a0734d428c99f3d041c55f5a6a/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:12.294380 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab357ad-fbce-4239-890b-22943bea0358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=929d0e35319f45babbf281bfbfb2c66f, user_id=067f97a0734d428c99f3d041c55f5a6a, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:12.297203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab357ad-fbce-4239-890b-22943bea0358 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:12.309232 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 937/1897] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:12 2022] PUT /identity/v3/projects/929d0e35319f45babbf281bfbfb2c66f/users/067f97a0734d428c99f3d041c55f5a6a/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:12.319582 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721df7c6-6ba9-4e82-a9b4-18db74d5faa7 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:12.319873 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721df7c6-6ba9-4e82-a9b4-18db74d5faa7 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:12.320171 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721df7c6-6ba9-4e82-a9b4-18db74d5faa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:12.323312 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f3620-9f6f-40b3-9152-4ebbc516df6c None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:12.351075 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-721df7c6-6ba9-4e82-a9b4-18db74d5faa7 None None] MFA Rules not processed for user `067f97a0734d428c99f3d041c55f5a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:12.352827 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-721df7c6-6ba9-4e82-a9b4-18db74d5faa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:12.383847 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d7114-2e50-4411-9c07-ed19f3ebeed2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:12.384662 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d7114-2e50-4411-9c07-ed19f3ebeed2 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:12.385147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d7114-2e50-4411-9c07-ed19f3ebeed2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:12.385489 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d7114-2e50-4411-9c07-ed19f3ebeed2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:12.419232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:12.450878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 938/1898] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:12.459557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721df7c6-6ba9-4e82-a9b4-18db74d5faa7 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:12.460618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-721df7c6-6ba9-4e82-a9b4-18db74d5faa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:12.466184 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d7114-2e50-4411-9c07-ed19f3ebeed2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:12.467723 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d7114-2e50-4411-9c07-ed19f3ebeed2 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:12.474035 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721df7c6-6ba9-4e82-a9b4-18db74d5faa7 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:12.482373 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 961/1899] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:13:12.482799 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37d500-1e53-4a2e-b862-1796aeed4bc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:12.483471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37d500-1e53-4a2e-b862-1796aeed4bc9 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:12.483755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37d500-1e53-4a2e-b862-1796aeed4bc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:12.484105 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37d500-1e53-4a2e-b862-1796aeed4bc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:12.494217 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37d500-1e53-4a2e-b862-1796aeed4bc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:12.495902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37d500-1e53-4a2e-b862-1796aeed4bc9 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:12.507103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 939/1900] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:12 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:12.517358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d7114-2e50-4411-9c07-ed19f3ebeed2 service cinder] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:12.589145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d7114-2e50-4411-9c07-ed19f3ebeed2 service cinder] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:12.593843 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f316a5f-e4f1-40ff-b94b-fc40c9a49253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:12.594788 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f316a5f-e4f1-40ff-b94b-fc40c9a49253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:12.595071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f316a5f-e4f1-40ff-b94b-fc40c9a49253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:12.595346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f316a5f-e4f1-40ff-b94b-fc40c9a49253 admin admin] PATH_INFO: `/v3/users/41a5bcad913143e59b114b163cb3f46c` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:12.596932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f316a5f-e4f1-40ff-b94b-fc40c9a49253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41a5bcad913143e59b114b163cb3f46c)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:12.599991 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f316a5f-e4f1-40ff-b94b-fc40c9a49253 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:12.655254 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-6f316a5f-e4f1-40ff-b94b-fc40c9a49253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41a5bcad913143e59b114b163cb3f46c'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:12.655958 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-6f316a5f-e4f1-40ff-b94b-fc40c9a49253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41a5bcad913143e59b114b163cb3f46c'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:12.656568 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-6f316a5f-e4f1-40ff-b94b-fc40c9a49253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41a5bcad913143e59b114b163cb3f46c'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:12.663489 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-6f316a5f-e4f1-40ff-b94b-fc40c9a49253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41a5bcad913143e59b114b163cb3f46c'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:12.676316 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-6f316a5f-e4f1-40ff-b94b-fc40c9a49253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41a5bcad913143e59b114b163cb3f46c'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:12.690418 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 962/1901] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:12 2022] DELETE /identity/v3/users/41a5bcad913143e59b114b163cb3f46c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:12.694903 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a18afd-0bee-490d-b026-4e101d389388 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:12.695192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a18afd-0bee-490d-b026-4e101d389388 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:12.695490 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a18afd-0bee-490d-b026-4e101d389388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:12.716410 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-74a18afd-0bee-490d-b026-4e101d389388 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:12.717634 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-74a18afd-0bee-490d-b026-4e101d389388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:12.793957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 940/1902] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:12.804061 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f316a5f-e4f1-40ff-b94b-fc40c9a49253 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:12.806354 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6f316a5f-e4f1-40ff-b94b-fc40c9a49253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:12.876827 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:12.878471 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f316a5f-e4f1-40ff-b94b-fc40c9a49253 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:12.885934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13531b2-a797-4f44-a508-563a9cb1947d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:12.887129 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13531b2-a797-4f44-a508-563a9cb1947d service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:12.887769 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13531b2-a797-4f44-a508-563a9cb1947d service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:12.888267 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13531b2-a797-4f44-a508-563a9cb1947d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:12.902840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13531b2-a797-4f44-a508-563a9cb1947d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:12.905009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13531b2-a797-4f44-a508-563a9cb1947d service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:12.924548 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 963/1903] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:13.274015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a18afd-0bee-490d-b026-4e101d389388 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:13.276484 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-74a18afd-0bee-490d-b026-4e101d389388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:13.367636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13531b2-a797-4f44-a508-563a9cb1947d service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:13.368571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a18afd-0bee-490d-b026-4e101d389388 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:13.372035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e13531b2-a797-4f44-a508-563a9cb1947d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:13.372780 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aff9c1-edaf-4386-95b3-fcbb6e6c4fb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:13.373457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aff9c1-edaf-4386-95b3-fcbb6e6c4fb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:13.373996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aff9c1-edaf-4386-95b3-fcbb6e6c4fb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:13.374354 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aff9c1-edaf-4386-95b3-fcbb6e6c4fb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:13.376530 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-57aff9c1-edaf-4386-95b3-fcbb6e6c4fb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:13.389075 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13531b2-a797-4f44-a508-563a9cb1947d service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:13.394355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8483b-eb2b-4561-a30f-de79e339a5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:13.394502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:13.394998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8483b-eb2b-4561-a30f-de79e339a5a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:13.395839 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8483b-eb2b-4561-a30f-de79e339a5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:13.403483 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8483b-eb2b-4561-a30f-de79e339a5a3 admin admin] PATH_INFO: `/v3/projects/ab05ecba3c9449bbb2071623d8fc4881` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:13.404923 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8483b-eb2b-4561-a30f-de79e339a5a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab05ecba3c9449bbb2071623d8fc4881)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:13.407095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8483b-eb2b-4561-a30f-de79e339a5a3 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:13.445402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aff9c1-edaf-4386-95b3-fcbb6e6c4fb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:13.446932 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aff9c1-edaf-4386-95b3-fcbb6e6c4fb7 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:13.458890 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-0ec8483b-eb2b-4561-a30f-de79e339a5a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab05ecba3c9449bbb2071623d8fc4881', 'request_id': 'req-0ec8483b-eb2b-4561-a30f-de79e339a5a3'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:13.465596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-0ec8483b-eb2b-4561-a30f-de79e339a5a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab05ecba3c9449bbb2071623d8fc4881', 'request_id': 'req-0ec8483b-eb2b-4561-a30f-de79e339a5a3'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:13.467755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 941/1904] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:13 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:13.474323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-0ec8483b-eb2b-4561-a30f-de79e339a5a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab05ecba3c9449bbb2071623d8fc4881', 'request_id': 'req-0ec8483b-eb2b-4561-a30f-de79e339a5a3'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:13.486845 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-0ec8483b-eb2b-4561-a30f-de79e339a5a3 admin admin] The token cache is being invalidate because project ab05ecba3c9449bbb2071623d8fc4881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:13.487159 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-0ec8483b-eb2b-4561-a30f-de79e339a5a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:13.487790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-0ec8483b-eb2b-4561-a30f-de79e339a5a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:13.489402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 964/1905] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:13 2022] DELETE /identity/v3/projects/ab05ecba3c9449bbb2071623d8fc4881 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:13.493644 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aff9c1-edaf-4386-95b3-fcbb6e6c4fb7 service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:13.506660 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aff9c1-edaf-4386-95b3-fcbb6e6c4fb7 service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:13.569158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71b56b-ca12-4a43-98c6-d47552113a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:13.569797 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71b56b-ca12-4a43-98c6-d47552113a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:13.570075 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71b56b-ca12-4a43-98c6-d47552113a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:13.570349 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71b56b-ca12-4a43-98c6-d47552113a3d admin admin] PATH_INFO: `/v3/users/7237e807baea4698a54c3f938dd8629a` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:13.572940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71b56b-ca12-4a43-98c6-d47552113a3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7237e807baea4698a54c3f938dd8629a)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:13.574551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71b56b-ca12-4a43-98c6-d47552113a3d admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:13.644700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-ab71b56b-ca12-4a43-98c6-d47552113a3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7237e807baea4698a54c3f938dd8629a'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:13.645504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-ab71b56b-ca12-4a43-98c6-d47552113a3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7237e807baea4698a54c3f938dd8629a'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:13.646154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-ab71b56b-ca12-4a43-98c6-d47552113a3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7237e807baea4698a54c3f938dd8629a'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:13.654955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-ab71b56b-ca12-4a43-98c6-d47552113a3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7237e807baea4698a54c3f938dd8629a'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:13.668852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-ab71b56b-ca12-4a43-98c6-d47552113a3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7237e807baea4698a54c3f938dd8629a'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:13.691750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 942/1906] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:13 2022] DELETE /identity/v3/users/7237e807baea4698a54c3f938dd8629a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:14.022055 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8483b-eb2b-4561-a30f-de79e339a5a3 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:14.023551 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-0ec8483b-eb2b-4561-a30f-de79e339a5a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:14.038642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8483b-eb2b-4561-a30f-de79e339a5a3 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:14.106758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31581b6-4642-4e96-952f-2d087d09036f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:14.107512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31581b6-4642-4e96-952f-2d087d09036f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:14.109154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31581b6-4642-4e96-952f-2d087d09036f admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:14.109154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31581b6-4642-4e96-952f-2d087d09036f admin admin] PATH_INFO: `/v3/projects/55a46ed7e8a34322995108bf5c16f5fa` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:14.109375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31581b6-4642-4e96-952f-2d087d09036f admin admin] RBAC: Authorizing `identity:delete_project(project_id=55a46ed7e8a34322995108bf5c16f5fa)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:14.111926 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31581b6-4642-4e96-952f-2d087d09036f admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:14.169021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d31581b6-4642-4e96-952f-2d087d09036f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55a46ed7e8a34322995108bf5c16f5fa', 'request_id': 'req-d31581b6-4642-4e96-952f-2d087d09036f'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:14.175870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d31581b6-4642-4e96-952f-2d087d09036f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55a46ed7e8a34322995108bf5c16f5fa', 'request_id': 'req-d31581b6-4642-4e96-952f-2d087d09036f'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:14.184840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d31581b6-4642-4e96-952f-2d087d09036f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55a46ed7e8a34322995108bf5c16f5fa', 'request_id': 'req-d31581b6-4642-4e96-952f-2d087d09036f'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:14.197646 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d31581b6-4642-4e96-952f-2d087d09036f admin admin] The token cache is being invalidate because project 55a46ed7e8a34322995108bf5c16f5fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:14.198195 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d31581b6-4642-4e96-952f-2d087d09036f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:14.198953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d31581b6-4642-4e96-952f-2d087d09036f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:14.200610 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 965/1907] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:14 2022] DELETE /identity/v3/projects/55a46ed7e8a34322995108bf5c16f5fa => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:14.205614 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f528e55-bec9-48eb-810c-b9b7fe013d5f None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:14.205901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f528e55-bec9-48eb-810c-b9b7fe013d5f None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:14.206179 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f528e55-bec9-48eb-810c-b9b7fe013d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:14.226701 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-6f528e55-bec9-48eb-810c-b9b7fe013d5f None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:14.228028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-6f528e55-bec9-48eb-810c-b9b7fe013d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:14.263959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:14.300875 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 943/1908] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:14.308939 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ab831-4929-49fe-b173-51e8810e26ea None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:14.309230 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ab831-4929-49fe-b173-51e8810e26ea None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:14.309535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ab831-4929-49fe-b173-51e8810e26ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:14.318637 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:14.334089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-3c5ab831-4929-49fe-b173-51e8810e26ea None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:14.336082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3c5ab831-4929-49fe-b173-51e8810e26ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:14.345358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 966/1909] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:14.349570 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f528e55-bec9-48eb-810c-b9b7fe013d5f None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:14.360181 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f528e55-bec9-48eb-810c-b9b7fe013d5f None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:14.365967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d1b1a-fd68-459a-9e80-f9b589a42ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:14.367088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d1b1a-fd68-459a-9e80-f9b589a42ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:14.367088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d1b1a-fd68-459a-9e80-f9b589a42ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:14.367402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d1b1a-fd68-459a-9e80-f9b589a42ba0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:14.368202 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d1b1a-fd68-459a-9e80-f9b589a42ba0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:14.369172 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d1b1a-fd68-459a-9e80-f9b589a42ba0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:14.371064 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d1b1a-fd68-459a-9e80-f9b589a42ba0 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:14.383134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 944/1910] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:14.388438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ab831-4929-49fe-b173-51e8810e26ea None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:14.399320 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ab831-4929-49fe-b173-51e8810e26ea None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:14.404497 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2a110-5894-4290-be57-e2eb354f7e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:14.405108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2a110-5894-4290-be57-e2eb354f7e6b admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:14.405386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2a110-5894-4290-be57-e2eb354f7e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:14.405656 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2a110-5894-4290-be57-e2eb354f7e6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:14.406348 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2a110-5894-4290-be57-e2eb354f7e6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:14.408354 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2a110-5894-4290-be57-e2eb354f7e6b admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:14.436938 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 967/1911] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:14 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:14.440890 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d1b1a-fd68-459a-9e80-f9b589a42ba0 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:14.452184 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d1b1a-fd68-459a-9e80-f9b589a42ba0 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:14.513759 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8841e-9dd0-497f-9784-b24ecc41391e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:14.514371 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8841e-9dd0-497f-9784-b24ecc41391e admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:14.514649 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8841e-9dd0-497f-9784-b24ecc41391e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:14.514921 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8841e-9dd0-497f-9784-b24ecc41391e admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:14.515586 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f8841e-9dd0-497f-9784-b24ecc41391e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:14.517557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f8841e-9dd0-497f-9784-b24ecc41391e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:14.540415 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 945/1912] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:14 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:14.545016 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2a110-5894-4290-be57-e2eb354f7e6b admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:14.556043 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2a110-5894-4290-be57-e2eb354f7e6b admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:14.561112 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802e61f-3e3e-4c52-831c-67c7a505333d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:14.561715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802e61f-3e3e-4c52-831c-67c7a505333d admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:14.561996 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802e61f-3e3e-4c52-831c-67c7a505333d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:14.562270 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802e61f-3e3e-4c52-831c-67c7a505333d admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:14.562916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802e61f-3e3e-4c52-831c-67c7a505333d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:14.564592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802e61f-3e3e-4c52-831c-67c7a505333d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:14.575068 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 968/1913] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:14.579246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8841e-9dd0-497f-9784-b24ecc41391e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:14.590572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8841e-9dd0-497f-9784-b24ecc41391e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:14.595831 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9c497-e856-44c4-a4f8-3288825ccdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:14.596486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9c497-e856-44c4-a4f8-3288825ccdbd admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:14.596773 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9c497-e856-44c4-a4f8-3288825ccdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:14.597041 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9c497-e856-44c4-a4f8-3288825ccdbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:14.597699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9c497-e856-44c4-a4f8-3288825ccdbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:14.599339 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9c497-e856-44c4-a4f8-3288825ccdbd admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:14.611094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 946/1914] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:14.615573 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802e61f-3e3e-4c52-831c-67c7a505333d admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:14.627619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802e61f-3e3e-4c52-831c-67c7a505333d admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:14.633733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad012f-c5a7-4427-941b-c72528fe036d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:14.634515 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad012f-c5a7-4427-941b-c72528fe036d admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:14.634900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad012f-c5a7-4427-941b-c72528fe036d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:14.635070 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad012f-c5a7-4427-941b-c72528fe036d admin admin] PATH_INFO: `/v3/projects/f190d8f03f614bb5b20780243b4e824f/users/26e27b02145546c69a8646cbd6a78dff/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:14.654104 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad012f-c5a7-4427-941b-c72528fe036d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f190d8f03f614bb5b20780243b4e824f, user_id=26e27b02145546c69a8646cbd6a78dff, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:14.656854 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad012f-c5a7-4427-941b-c72528fe036d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:14.672385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 969/1915] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:14 2022] PUT /identity/v3/projects/f190d8f03f614bb5b20780243b4e824f/users/26e27b02145546c69a8646cbd6a78dff/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:14.683865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702a038-c5c6-4e68-8dcf-63d12a5c1d1d None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:14.684149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702a038-c5c6-4e68-8dcf-63d12a5c1d1d None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:14.684420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702a038-c5c6-4e68-8dcf-63d12a5c1d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:14.713460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-0702a038-c5c6-4e68-8dcf-63d12a5c1d1d None None] MFA Rules not processed for user `26e27b02145546c69a8646cbd6a78dff`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:14.714742 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-0702a038-c5c6-4e68-8dcf-63d12a5c1d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:14.789478 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 947/1916] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:14.801186 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad012f-c5a7-4427-941b-c72528fe036d admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:14.804999 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-33ad012f-c5a7-4427-941b-c72528fe036d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:14.873719 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad012f-c5a7-4427-941b-c72528fe036d admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:14.877469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2313870-106e-4dc3-a223-e0d8b023ee44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:14.878181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2313870-106e-4dc3-a223-e0d8b023ee44 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:14.878464 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2313870-106e-4dc3-a223-e0d8b023ee44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:14.878737 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2313870-106e-4dc3-a223-e0d8b023ee44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:14.888302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2313870-106e-4dc3-a223-e0d8b023ee44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:14.889777 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2313870-106e-4dc3-a223-e0d8b023ee44 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:14.898850 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 970/1917] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:14.918673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702a038-c5c6-4e68-8dcf-63d12a5c1d1d None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:14.954771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:14.985586 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702a038-c5c6-4e68-8dcf-63d12a5c1d1d None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:14.991092 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfec31-835f-4f78-9823-110088b1807e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:14.991863 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfec31-835f-4f78-9823-110088b1807e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:14.992140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfec31-835f-4f78-9823-110088b1807e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:14.992408 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfec31-835f-4f78-9823-110088b1807e admin admin] PATH_INFO: `/v3/users/26e27b02145546c69a8646cbd6a78dff` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:14.993874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfec31-835f-4f78-9823-110088b1807e admin admin] RBAC: Authorizing `identity:delete_user(user_id=26e27b02145546c69a8646cbd6a78dff)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:14.995917 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfec31-835f-4f78-9823-110088b1807e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:15.046716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-74cfec31-835f-4f78-9823-110088b1807e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26e27b02145546c69a8646cbd6a78dff'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:15.047377 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-74cfec31-835f-4f78-9823-110088b1807e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26e27b02145546c69a8646cbd6a78dff'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:15.048080 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-74cfec31-835f-4f78-9823-110088b1807e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26e27b02145546c69a8646cbd6a78dff'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:15.054839 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-74cfec31-835f-4f78-9823-110088b1807e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26e27b02145546c69a8646cbd6a78dff'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:15.066529 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-74cfec31-835f-4f78-9823-110088b1807e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26e27b02145546c69a8646cbd6a78dff'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:15.080485 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 948/1918] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:14 2022] DELETE /identity/v3/users/26e27b02145546c69a8646cbd6a78dff => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:15.085434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47660c-cc4e-4ec5-9d9b-0abc7fce9117 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:15.085798 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47660c-cc4e-4ec5-9d9b-0abc7fce9117 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:15.086143 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47660c-cc4e-4ec5-9d9b-0abc7fce9117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:15.106376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-8c47660c-cc4e-4ec5-9d9b-0abc7fce9117 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:15.107622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8c47660c-cc4e-4ec5-9d9b-0abc7fce9117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:15.166874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 971/1919] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:15.176751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfec31-835f-4f78-9823-110088b1807e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:15.178462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-74cfec31-835f-4f78-9823-110088b1807e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:15.239245 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfec31-835f-4f78-9823-110088b1807e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:15.243845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785aa228-6d60-4d8e-95ca-314e49db4e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:15.244484 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aa228-6d60-4d8e-95ca-314e49db4e2b service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:15.244762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aa228-6d60-4d8e-95ca-314e49db4e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:15.245019 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aa228-6d60-4d8e-95ca-314e49db4e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:15.255052 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785aa228-6d60-4d8e-95ca-314e49db4e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:15.256520 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785aa228-6d60-4d8e-95ca-314e49db4e2b service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:15.266796 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 949/1920] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:15.605452 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47660c-cc4e-4ec5-9d9b-0abc7fce9117 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:15.607085 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8c47660c-cc4e-4ec5-9d9b-0abc7fce9117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:15.620015 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47660c-cc4e-4ec5-9d9b-0abc7fce9117 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:15.625062 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00a482-d279-46ca-98dc-81a0d357bccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:15.625708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00a482-d279-46ca-98dc-81a0d357bccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:15.625990 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00a482-d279-46ca-98dc-81a0d357bccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:15.626266 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00a482-d279-46ca-98dc-81a0d357bccd admin admin] PATH_INFO: `/v3/projects/f190d8f03f614bb5b20780243b4e824f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:15.627468 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00a482-d279-46ca-98dc-81a0d357bccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f190d8f03f614bb5b20780243b4e824f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:15.629976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00a482-d279-46ca-98dc-81a0d357bccd admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:15.679013 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:15.701321 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5b00a482-d279-46ca-98dc-81a0d357bccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f190d8f03f614bb5b20780243b4e824f', 'request_id': 'req-5b00a482-d279-46ca-98dc-81a0d357bccd'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:15.709537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5b00a482-d279-46ca-98dc-81a0d357bccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f190d8f03f614bb5b20780243b4e824f', 'request_id': 'req-5b00a482-d279-46ca-98dc-81a0d357bccd'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:15.720881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5b00a482-d279-46ca-98dc-81a0d357bccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f190d8f03f614bb5b20780243b4e824f', 'request_id': 'req-5b00a482-d279-46ca-98dc-81a0d357bccd'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:15.736474 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5b00a482-d279-46ca-98dc-81a0d357bccd admin admin] The token cache is being invalidate because project f190d8f03f614bb5b20780243b4e824f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:15.736778 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5b00a482-d279-46ca-98dc-81a0d357bccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:15.737469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5b00a482-d279-46ca-98dc-81a0d357bccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:15.739277 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 972/1921] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:15 2022] DELETE /identity/v3/projects/f190d8f03f614bb5b20780243b4e824f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:15.744322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cac7c0-714a-4dc2-bae9-479b4f5e17c3 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:15.744771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cac7c0-714a-4dc2-bae9-479b4f5e17c3 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:15.744868 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cac7c0-714a-4dc2-bae9-479b4f5e17c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:15.768066 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-06cac7c0-714a-4dc2-bae9-479b4f5e17c3 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:15.769330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-06cac7c0-714a-4dc2-bae9-479b4f5e17c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:15.820595 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:15.837220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 950/1922] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:15.846482 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad703c-2dc3-43aa-836a-4905fb6127c0 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:15.846770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad703c-2dc3-43aa-836a-4905fb6127c0 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:15.847046 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad703c-2dc3-43aa-836a-4905fb6127c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:15.867202 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-35ad703c-2dc3-43aa-836a-4905fb6127c0 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:15.868478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-35ad703c-2dc3-43aa-836a-4905fb6127c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:15.876604 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 973/1923] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:15.881058 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cac7c0-714a-4dc2-bae9-479b4f5e17c3 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:15.890899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cac7c0-714a-4dc2-bae9-479b4f5e17c3 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:15.895957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284b2fc-bcc3-4aaa-a6c7-bad60cd7e3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:15.896570 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284b2fc-bcc3-4aaa-a6c7-bad60cd7e3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:15.896849 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284b2fc-bcc3-4aaa-a6c7-bad60cd7e3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:15.897189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284b2fc-bcc3-4aaa-a6c7-bad60cd7e3f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:15.897939 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284b2fc-bcc3-4aaa-a6c7-bad60cd7e3f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:15.898222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284b2fc-bcc3-4aaa-a6c7-bad60cd7e3f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:15.899916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284b2fc-bcc3-4aaa-a6c7-bad60cd7e3f8 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:15.912710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 951/1924] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:15.916623 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad703c-2dc3-43aa-836a-4905fb6127c0 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:15.926110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad703c-2dc3-43aa-836a-4905fb6127c0 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:15.931030 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1fab0-43c9-4168-9fb4-87be614ca681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:15.931852 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1fab0-43c9-4168-9fb4-87be614ca681 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:15.932132 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1fab0-43c9-4168-9fb4-87be614ca681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:15.932403 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1fab0-43c9-4168-9fb4-87be614ca681 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:15.933045 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1fab0-43c9-4168-9fb4-87be614ca681 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:15.934981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1fab0-43c9-4168-9fb4-87be614ca681 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:15.969814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 974/1925] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:15 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:15.971019 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284b2fc-bcc3-4aaa-a6c7-bad60cd7e3f8 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:15.986950 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284b2fc-bcc3-4aaa-a6c7-bad60cd7e3f8 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:16.078142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17bb1c-80bc-47b2-994e-faa79697ed01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:16.078773 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17bb1c-80bc-47b2-994e-faa79697ed01 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:16.079051 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17bb1c-80bc-47b2-994e-faa79697ed01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:16.079512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17bb1c-80bc-47b2-994e-faa79697ed01 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:16.080151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17bb1c-80bc-47b2-994e-faa79697ed01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:16.082399 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17bb1c-80bc-47b2-994e-faa79697ed01 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:16.108016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 952/1926] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:15 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:16.112025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1fab0-43c9-4168-9fb4-87be614ca681 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:16.123970 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1fab0-43c9-4168-9fb4-87be614ca681 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:16.129836 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5f046-2ed2-4514-b129-545ade19736c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:16.130452 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5f046-2ed2-4514-b129-545ade19736c admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:16.130729 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5f046-2ed2-4514-b129-545ade19736c admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:16.130997 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5f046-2ed2-4514-b129-545ade19736c admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:16.131786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5f046-2ed2-4514-b129-545ade19736c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:16.134379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5f046-2ed2-4514-b129-545ade19736c admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:16.146723 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 975/1927] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:16.151192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17bb1c-80bc-47b2-994e-faa79697ed01 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:16.161754 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17bb1c-80bc-47b2-994e-faa79697ed01 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:16.166622 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1a307-4ad1-4285-947b-d9f83f8f9824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:16.167189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1a307-4ad1-4285-947b-d9f83f8f9824 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:16.167500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1a307-4ad1-4285-947b-d9f83f8f9824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:16.167777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1a307-4ad1-4285-947b-d9f83f8f9824 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:16.168428 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1a307-4ad1-4285-947b-d9f83f8f9824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:16.170364 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1a307-4ad1-4285-947b-d9f83f8f9824 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:16.182347 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 953/1928] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:16.187552 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5f046-2ed2-4514-b129-545ade19736c admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:16.202443 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5f046-2ed2-4514-b129-545ade19736c admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:16.207839 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08243ae2-eaf7-4a80-87d6-681a38b25b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:16.208563 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08243ae2-eaf7-4a80-87d6-681a38b25b20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:16.208843 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08243ae2-eaf7-4a80-87d6-681a38b25b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:16.209109 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08243ae2-eaf7-4a80-87d6-681a38b25b20 admin admin] PATH_INFO: `/v3/projects/21ed2d9c51f44fd09743b5bfc408e240/users/139e62ed66de4aa2a5874b3907501670/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:16.224686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08243ae2-eaf7-4a80-87d6-681a38b25b20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ed2d9c51f44fd09743b5bfc408e240, user_id=139e62ed66de4aa2a5874b3907501670, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:16.228032 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08243ae2-eaf7-4a80-87d6-681a38b25b20 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:16.243391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 976/1929] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:16 2022] PUT /identity/v3/projects/21ed2d9c51f44fd09743b5bfc408e240/users/139e62ed66de4aa2a5874b3907501670/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:16.253910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbab968-9baa-44ee-9ec3-5fa2d68ea030 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:16.254194 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbab968-9baa-44ee-9ec3-5fa2d68ea030 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:16.254467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbab968-9baa-44ee-9ec3-5fa2d68ea030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:16.287799 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-9fbab968-9baa-44ee-9ec3-5fa2d68ea030 None None] MFA Rules not processed for user `139e62ed66de4aa2a5874b3907501670`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:16.289033 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9fbab968-9baa-44ee-9ec3-5fa2d68ea030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:16.360118 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 954/1930] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:16.379597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08243ae2-eaf7-4a80-87d6-681a38b25b20 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:16.382594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-08243ae2-eaf7-4a80-87d6-681a38b25b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:16.452143 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08243ae2-eaf7-4a80-87d6-681a38b25b20 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:16.457467 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933cfc6-af6d-4b0e-89a2-30c0d7d1f44a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:16.458477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933cfc6-af6d-4b0e-89a2-30c0d7d1f44a service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:16.458949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933cfc6-af6d-4b0e-89a2-30c0d7d1f44a service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:16.459250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933cfc6-af6d-4b0e-89a2-30c0d7d1f44a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:16.470285 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933cfc6-af6d-4b0e-89a2-30c0d7d1f44a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:16.471932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933cfc6-af6d-4b0e-89a2-30c0d7d1f44a service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:16.482508 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 977/1931] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:21.898755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbab968-9baa-44ee-9ec3-5fa2d68ea030 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:21.900333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9fbab968-9baa-44ee-9ec3-5fa2d68ea030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:21.912683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbab968-9baa-44ee-9ec3-5fa2d68ea030 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:21.918514 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4b263-47b1-4d68-873e-66c3c00cb81c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:21.919300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4b263-47b1-4d68-873e-66c3c00cb81c service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:21.919691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4b263-47b1-4d68-873e-66c3c00cb81c service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:21.920034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4b263-47b1-4d68-873e-66c3c00cb81c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:21.922852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-16a4b263-47b1-4d68-873e-66c3c00cb81c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:21.948143 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933cfc6-af6d-4b0e-89a2-30c0d7d1f44a service cinder] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:21.950256 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-2933cfc6-af6d-4b0e-89a2-30c0d7d1f44a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:21.988005 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:22.006331 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4b263-47b1-4d68-873e-66c3c00cb81c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:22.008525 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4b263-47b1-4d68-873e-66c3c00cb81c service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:22.018733 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 955/1932] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:22.062271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933cfc6-af6d-4b0e-89a2-30c0d7d1f44a service cinder] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:22.067392 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531c6c4-a1ca-4ad6-9274-ab2390948ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:22.068201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531c6c4-a1ca-4ad6-9274-ab2390948ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:22.068486 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531c6c4-a1ca-4ad6-9274-ab2390948ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:22.068761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531c6c4-a1ca-4ad6-9274-ab2390948ede admin admin] PATH_INFO: `/v3/users/ae739cd6305148cc9f99afe62f9079c7` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:22.069900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531c6c4-a1ca-4ad6-9274-ab2390948ede admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae739cd6305148cc9f99afe62f9079c7)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:22.075121 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531c6c4-a1ca-4ad6-9274-ab2390948ede admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:22.082275 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:22.148436 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1531c6c4-a1ca-4ad6-9274-ab2390948ede admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae739cd6305148cc9f99afe62f9079c7'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:22.149386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1531c6c4-a1ca-4ad6-9274-ab2390948ede admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae739cd6305148cc9f99afe62f9079c7'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:22.150557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1531c6c4-a1ca-4ad6-9274-ab2390948ede admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae739cd6305148cc9f99afe62f9079c7'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:22.172854 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1531c6c4-a1ca-4ad6-9274-ab2390948ede admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae739cd6305148cc9f99afe62f9079c7'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:22.201433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1531c6c4-a1ca-4ad6-9274-ab2390948ede admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae739cd6305148cc9f99afe62f9079c7'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:22.219819 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 978/1933] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:21 2022] DELETE /identity/v3/users/ae739cd6305148cc9f99afe62f9079c7 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:22.224706 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373d0d9-217f-406f-bbda-0eb20e6d0ba9 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:22.224706 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373d0d9-217f-406f-bbda-0eb20e6d0ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:22.225129 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373d0d9-217f-406f-bbda-0eb20e6d0ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:22.249847 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-a373d0d9-217f-406f-bbda-0eb20e6d0ba9 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:22.251284 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-a373d0d9-217f-406f-bbda-0eb20e6d0ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:22.351720 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 956/1934] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:22.365198 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531c6c4-a1ca-4ad6-9274-ab2390948ede admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:22.366644 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-1531c6c4-a1ca-4ad6-9274-ab2390948ede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:22.445665 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531c6c4-a1ca-4ad6-9274-ab2390948ede admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:22.449835 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7aab9-16e5-4c8e-89c4-dc42f0444f95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:22.450818 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7aab9-16e5-4c8e-89c4-dc42f0444f95 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:22.451824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7aab9-16e5-4c8e-89c4-dc42f0444f95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:22.452336 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7aab9-16e5-4c8e-89c4-dc42f0444f95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:22.465884 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7aab9-16e5-4c8e-89c4-dc42f0444f95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:22.467337 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7aab9-16e5-4c8e-89c4-dc42f0444f95 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:22.492736 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 979/1935] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:22.820122 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373d0d9-217f-406f-bbda-0eb20e6d0ba9 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:22.822106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-a373d0d9-217f-406f-bbda-0eb20e6d0ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:22.850631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373d0d9-217f-406f-bbda-0eb20e6d0ba9 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:22.857393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79fbad-00ea-475f-a46a-fd4574f96026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:22.858284 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79fbad-00ea-475f-a46a-fd4574f96026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:22.858709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79fbad-00ea-475f-a46a-fd4574f96026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:22.859123 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79fbad-00ea-475f-a46a-fd4574f96026 admin admin] PATH_INFO: `/v3/projects/afccf25909bb4f84a61397e1bf6d87cd` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:22.870852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79fbad-00ea-475f-a46a-fd4574f96026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afccf25909bb4f84a61397e1bf6d87cd)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:22.873627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79fbad-00ea-475f-a46a-fd4574f96026 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:22.944386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1a79fbad-00ea-475f-a46a-fd4574f96026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afccf25909bb4f84a61397e1bf6d87cd', 'request_id': 'req-1a79fbad-00ea-475f-a46a-fd4574f96026'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:22.969701 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1a79fbad-00ea-475f-a46a-fd4574f96026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afccf25909bb4f84a61397e1bf6d87cd', 'request_id': 'req-1a79fbad-00ea-475f-a46a-fd4574f96026'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:22.984239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1a79fbad-00ea-475f-a46a-fd4574f96026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afccf25909bb4f84a61397e1bf6d87cd', 'request_id': 'req-1a79fbad-00ea-475f-a46a-fd4574f96026'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:23.005789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1a79fbad-00ea-475f-a46a-fd4574f96026 admin admin] The token cache is being invalidate because project afccf25909bb4f84a61397e1bf6d87cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:23.006302 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1a79fbad-00ea-475f-a46a-fd4574f96026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:23.007165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1a79fbad-00ea-475f-a46a-fd4574f96026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:23.013167 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 957/1936] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:22 2022] DELETE /identity/v3/projects/afccf25909bb4f84a61397e1bf6d87cd => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:23.014828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7aab9-16e5-4c8e-89c4-dc42f0444f95 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:23.040288 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7aab9-16e5-4c8e-89c4-dc42f0444f95 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:23.157398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf740052-42f1-4518-956b-702880129065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:23.158174 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf740052-42f1-4518-956b-702880129065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:23.158518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf740052-42f1-4518-956b-702880129065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:23.158851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf740052-42f1-4518-956b-702880129065 admin admin] PATH_INFO: `/v3/users/dcbee6a4f7e649fabbf63b1c9c7bfa24` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:23.160130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf740052-42f1-4518-956b-702880129065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcbee6a4f7e649fabbf63b1c9c7bfa24)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:23.164264 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf740052-42f1-4518-956b-702880129065 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:23.175726 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79fbad-00ea-475f-a46a-fd4574f96026 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:23.177588 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-1a79fbad-00ea-475f-a46a-fd4574f96026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:23.193334 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79fbad-00ea-475f-a46a-fd4574f96026 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:23.202168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31113c3d-e01e-40c7-ac49-5ab1f4b0039e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:23.203565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31113c3d-e01e-40c7-ac49-5ab1f4b0039e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:23.207533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:23.208308 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31113c3d-e01e-40c7-ac49-5ab1f4b0039e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:23.208434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31113c3d-e01e-40c7-ac49-5ab1f4b0039e admin admin] PATH_INFO: `/v3/users/7d1401598f1b4cf7b7c3e89dfbd0debf` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:23.211304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31113c3d-e01e-40c7-ac49-5ab1f4b0039e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d1401598f1b4cf7b7c3e89dfbd0debf)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:23.214613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31113c3d-e01e-40c7-ac49-5ab1f4b0039e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:23.239699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-bf740052-42f1-4518-956b-702880129065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcbee6a4f7e649fabbf63b1c9c7bfa24'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:23.240742 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-bf740052-42f1-4518-956b-702880129065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcbee6a4f7e649fabbf63b1c9c7bfa24'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:23.241997 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-bf740052-42f1-4518-956b-702880129065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcbee6a4f7e649fabbf63b1c9c7bfa24'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:23.251986 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-bf740052-42f1-4518-956b-702880129065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcbee6a4f7e649fabbf63b1c9c7bfa24'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:23.271661 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-bf740052-42f1-4518-956b-702880129065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcbee6a4f7e649fabbf63b1c9c7bfa24'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:23.280176 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-31113c3d-e01e-40c7-ac49-5ab1f4b0039e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d1401598f1b4cf7b7c3e89dfbd0debf'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:23.281155 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-31113c3d-e01e-40c7-ac49-5ab1f4b0039e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d1401598f1b4cf7b7c3e89dfbd0debf'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:23.282222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-31113c3d-e01e-40c7-ac49-5ab1f4b0039e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d1401598f1b4cf7b7c3e89dfbd0debf'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:23.294034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-31113c3d-e01e-40c7-ac49-5ab1f4b0039e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d1401598f1b4cf7b7c3e89dfbd0debf'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:23.303251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 980/1937] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:23 2022] DELETE /identity/v3/users/dcbee6a4f7e649fabbf63b1c9c7bfa24 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:23.304290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:23.311343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-31113c3d-e01e-40c7-ac49-5ab1f4b0039e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d1401598f1b4cf7b7c3e89dfbd0debf'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:23.335200 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 958/1938] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:23 2022] DELETE /identity/v3/users/7d1401598f1b4cf7b7c3e89dfbd0debf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:23.339963 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba80c89-1b94-4330-a786-fa928a89f11a None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:23.340248 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba80c89-1b94-4330-a786-fa928a89f11a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:23.340525 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba80c89-1b94-4330-a786-fa928a89f11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:23.364531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-9ba80c89-1b94-4330-a786-fa928a89f11a None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:23.365863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9ba80c89-1b94-4330-a786-fa928a89f11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:23.433613 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 981/1939] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:23.444077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31113c3d-e01e-40c7-ac49-5ab1f4b0039e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:23.445497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-31113c3d-e01e-40c7-ac49-5ab1f4b0039e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:23.518415 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31113c3d-e01e-40c7-ac49-5ab1f4b0039e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:23.522298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a23a0b-4c6a-4326-a788-61569c7ce5bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:23.523100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a23a0b-4c6a-4326-a788-61569c7ce5bb service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:23.523495 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a23a0b-4c6a-4326-a788-61569c7ce5bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:23.524098 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a23a0b-4c6a-4326-a788-61569c7ce5bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:23.542211 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a23a0b-4c6a-4326-a788-61569c7ce5bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:23.544243 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a23a0b-4c6a-4326-a788-61569c7ce5bb service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:23.556281 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 959/1940] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:23.920847 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba80c89-1b94-4330-a786-fa928a89f11a None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:23.924649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9ba80c89-1b94-4330-a786-fa928a89f11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:23.937726 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba80c89-1b94-4330-a786-fa928a89f11a None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:23.948160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf660b-516d-46c1-9945-94ad6ae6d126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:23.948934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf660b-516d-46c1-9945-94ad6ae6d126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:23.949319 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf660b-516d-46c1-9945-94ad6ae6d126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:23.949712 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf660b-516d-46c1-9945-94ad6ae6d126 admin admin] PATH_INFO: `/v3/projects/2ae0afc91d574af79906ec9db97324b7` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:23.952776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf660b-516d-46c1-9945-94ad6ae6d126 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ae0afc91d574af79906ec9db97324b7)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:23.955126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf660b-516d-46c1-9945-94ad6ae6d126 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:24.032378 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-6eaf660b-516d-46c1-9945-94ad6ae6d126 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ae0afc91d574af79906ec9db97324b7', 'request_id': 'req-6eaf660b-516d-46c1-9945-94ad6ae6d126'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:24.039247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-6eaf660b-516d-46c1-9945-94ad6ae6d126 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ae0afc91d574af79906ec9db97324b7', 'request_id': 'req-6eaf660b-516d-46c1-9945-94ad6ae6d126'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:24.049150 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-6eaf660b-516d-46c1-9945-94ad6ae6d126 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ae0afc91d574af79906ec9db97324b7', 'request_id': 'req-6eaf660b-516d-46c1-9945-94ad6ae6d126'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:24.053383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a23a0b-4c6a-4326-a788-61569c7ce5bb service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:24.055248 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-73a23a0b-4c6a-4326-a788-61569c7ce5bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:24.062513 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-6eaf660b-516d-46c1-9945-94ad6ae6d126 admin admin] The token cache is being invalidate because project 2ae0afc91d574af79906ec9db97324b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:24.062810 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-6eaf660b-516d-46c1-9945-94ad6ae6d126 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:24.063512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-6eaf660b-516d-46c1-9945-94ad6ae6d126 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:24.065202 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 982/1941] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:23 2022] DELETE /identity/v3/projects/2ae0afc91d574af79906ec9db97324b7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:24.070378 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680de4d-0532-4b16-8ad8-df55e9566315 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:24.071196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680de4d-0532-4b16-8ad8-df55e9566315 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:24.072812 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680de4d-0532-4b16-8ad8-df55e9566315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:24.074814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a23a0b-4c6a-4326-a788-61569c7ce5bb service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:24.110050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-6680de4d-0532-4b16-8ad8-df55e9566315 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:24.111299 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6680de4d-0532-4b16-8ad8-df55e9566315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:24.170370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:24.198097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142ff3d-17d1-4d7b-9d1c-f14d36876537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:24.199372 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142ff3d-17d1-4d7b-9d1c-f14d36876537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:24.199855 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142ff3d-17d1-4d7b-9d1c-f14d36876537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:24.200368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142ff3d-17d1-4d7b-9d1c-f14d36876537 admin admin] PATH_INFO: `/v3/projects/3b1d47921fb14648b8da10af27d12ce4` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:24.201948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5142ff3d-17d1-4d7b-9d1c-f14d36876537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b1d47921fb14648b8da10af27d12ce4)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:24.204940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5142ff3d-17d1-4d7b-9d1c-f14d36876537 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:24.268680 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 983/1942] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:24.278569 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92dbc5e-2b75-4c96-88bf-1b7bae063698 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:24.278859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92dbc5e-2b75-4c96-88bf-1b7bae063698 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:24.279140 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92dbc5e-2b75-4c96-88bf-1b7bae063698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:24.319864 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-b92dbc5e-2b75-4c96-88bf-1b7bae063698 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:24.321719 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b92dbc5e-2b75-4c96-88bf-1b7bae063698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:24.352585 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-5142ff3d-17d1-4d7b-9d1c-f14d36876537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b1d47921fb14648b8da10af27d12ce4', 'request_id': 'req-5142ff3d-17d1-4d7b-9d1c-f14d36876537'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:24.369671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-5142ff3d-17d1-4d7b-9d1c-f14d36876537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b1d47921fb14648b8da10af27d12ce4', 'request_id': 'req-5142ff3d-17d1-4d7b-9d1c-f14d36876537'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:24.387077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-5142ff3d-17d1-4d7b-9d1c-f14d36876537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b1d47921fb14648b8da10af27d12ce4', 'request_id': 'req-5142ff3d-17d1-4d7b-9d1c-f14d36876537'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:24.412417 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-5142ff3d-17d1-4d7b-9d1c-f14d36876537 admin admin] The token cache is being invalidate because project 3b1d47921fb14648b8da10af27d12ce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:24.412722 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-5142ff3d-17d1-4d7b-9d1c-f14d36876537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:24.413408 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-5142ff3d-17d1-4d7b-9d1c-f14d36876537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:24.417477 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 960/1943] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:24 2022] DELETE /identity/v3/projects/3b1d47921fb14648b8da10af27d12ce4 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:24.425326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142ff3d-17d1-4d7b-9d1c-f14d36876537 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:24.433517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-5142ff3d-17d1-4d7b-9d1c-f14d36876537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:24.467664 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 984/1944] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:24.478173 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:24.488404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92dbc5e-2b75-4c96-88bf-1b7bae063698 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:24.518367 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92dbc5e-2b75-4c96-88bf-1b7bae063698 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:24.529370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bd26d-d410-4ccd-a57a-b0ed42d16900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:24.530636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bd26d-d410-4ccd-a57a-b0ed42d16900 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:24.533488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bd26d-d410-4ccd-a57a-b0ed42d16900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:24.533996 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bd26d-d410-4ccd-a57a-b0ed42d16900 admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:24.534503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0bd26d-d410-4ccd-a57a-b0ed42d16900 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:24.539844 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0bd26d-d410-4ccd-a57a-b0ed42d16900 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:24.541825 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0bd26d-d410-4ccd-a57a-b0ed42d16900 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:24.567204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 985/1945] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:24.581125 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bd26d-d410-4ccd-a57a-b0ed42d16900 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:24.595542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bd26d-d410-4ccd-a57a-b0ed42d16900 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:24.600700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bfe61-0f17-4817-80dd-59cc7b26ff96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:24.601903 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bfe61-0f17-4817-80dd-59cc7b26ff96 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:24.601903 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bfe61-0f17-4817-80dd-59cc7b26ff96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:24.602089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bfe61-0f17-4817-80dd-59cc7b26ff96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:24.602707 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4bfe61-0f17-4817-80dd-59cc7b26ff96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:24.605003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4bfe61-0f17-4817-80dd-59cc7b26ff96 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:24.627258 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142ff3d-17d1-4d7b-9d1c-f14d36876537 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:24.640165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac028b-87f6-43a0-b4ef-285d9c8de7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:24.640861 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac028b-87f6-43a0-b4ef-285d9c8de7a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:24.641146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac028b-87f6-43a0-b4ef-285d9c8de7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:24.641429 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac028b-87f6-43a0-b4ef-285d9c8de7a5 admin admin] PATH_INFO: `/v3/users/6bb4e0fbf23341c1a900aab2e87db9e5` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:24.644098 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac028b-87f6-43a0-b4ef-285d9c8de7a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bb4e0fbf23341c1a900aab2e87db9e5)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:24.646104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac028b-87f6-43a0-b4ef-285d9c8de7a5 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:24.659042 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 986/1946] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:24 2022] POST /identity/v3/projects => generated 379 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:24.663125 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bfe61-0f17-4817-80dd-59cc7b26ff96 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:24.760856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-13ac028b-87f6-43a0-b4ef-285d9c8de7a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bb4e0fbf23341c1a900aab2e87db9e5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:24.761552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-13ac028b-87f6-43a0-b4ef-285d9c8de7a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bb4e0fbf23341c1a900aab2e87db9e5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:24.762768 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-13ac028b-87f6-43a0-b4ef-285d9c8de7a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bb4e0fbf23341c1a900aab2e87db9e5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:24.789288 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-13ac028b-87f6-43a0-b4ef-285d9c8de7a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bb4e0fbf23341c1a900aab2e87db9e5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:24.791875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bfe61-0f17-4817-80dd-59cc7b26ff96 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:24.815910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-13ac028b-87f6-43a0-b4ef-285d9c8de7a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bb4e0fbf23341c1a900aab2e87db9e5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:24.816873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d67c68-092c-4a9d-8b66-131c1a8d1ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:24.819134 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d67c68-092c-4a9d-8b66-131c1a8d1ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:24.819655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d67c68-092c-4a9d-8b66-131c1a8d1ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:24.820669 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d67c68-092c-4a9d-8b66-131c1a8d1ae2 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:24.826246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d67c68-092c-4a9d-8b66-131c1a8d1ae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:24.828937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d67c68-092c-4a9d-8b66-131c1a8d1ae2 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:24.840222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 961/1947] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:24 2022] DELETE /identity/v3/users/6bb4e0fbf23341c1a900aab2e87db9e5 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:24.875358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 987/1948] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:24 2022] POST /identity/v3/users => generated 290 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:24.876377 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac028b-87f6-43a0-b4ef-285d9c8de7a5 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:24.878367 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-13ac028b-87f6-43a0-b4ef-285d9c8de7a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:24.966194 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d67c68-092c-4a9d-8b66-131c1a8d1ae2 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:24.967897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-44d67c68-092c-4a9d-8b66-131c1a8d1ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:24.976214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac028b-87f6-43a0-b4ef-285d9c8de7a5 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:24.981916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c4cf7-58b3-4265-b782-34e0bd02430c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:24.983482 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c4cf7-58b3-4265-b782-34e0bd02430c admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:24.983914 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c4cf7-58b3-4265-b782-34e0bd02430c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:24.984462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d67c68-092c-4a9d-8b66-131c1a8d1ae2 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:24.985004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c4cf7-58b3-4265-b782-34e0bd02430c admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:24.986091 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c4cf7-58b3-4265-b782-34e0bd02430c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:24.988916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c4cf7-58b3-4265-b782-34e0bd02430c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:24.989876 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4af5f9-fd47-4aca-9407-ddfb63200825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:24.991373 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4af5f9-fd47-4aca-9407-ddfb63200825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:24.991373 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4af5f9-fd47-4aca-9407-ddfb63200825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:24.992181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4af5f9-fd47-4aca-9407-ddfb63200825 admin admin] PATH_INFO: `/v3/users/e206598ef86043ffbe5e63c8b5d80cd8` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:24.994055 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4af5f9-fd47-4aca-9407-ddfb63200825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e206598ef86043ffbe5e63c8b5d80cd8)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:24.996547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4af5f9-fd47-4aca-9407-ddfb63200825 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:25.038592 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 962/1949] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:25.043864 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c4cf7-58b3-4265-b782-34e0bd02430c admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:25.060385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:25.075346 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c4cf7-58b3-4265-b782-34e0bd02430c admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:25.086499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e21a5b-adff-40e0-aa2b-9bc472509de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:25.087881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e21a5b-adff-40e0-aa2b-9bc472509de2 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:25.088560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e21a5b-adff-40e0-aa2b-9bc472509de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:25.088975 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e21a5b-adff-40e0-aa2b-9bc472509de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:25.090113 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e21a5b-adff-40e0-aa2b-9bc472509de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:25.098482 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e21a5b-adff-40e0-aa2b-9bc472509de2 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:25.115244 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 963/1950] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:25.119991 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e21a5b-adff-40e0-aa2b-9bc472509de2 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:25.134515 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fb4af5f9-fd47-4aca-9407-ddfb63200825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e206598ef86043ffbe5e63c8b5d80cd8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:25.135398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fb4af5f9-fd47-4aca-9407-ddfb63200825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e206598ef86043ffbe5e63c8b5d80cd8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:25.136203 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fb4af5f9-fd47-4aca-9407-ddfb63200825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e206598ef86043ffbe5e63c8b5d80cd8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:25.149514 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e21a5b-adff-40e0-aa2b-9bc472509de2 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:25.150952 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fb4af5f9-fd47-4aca-9407-ddfb63200825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e206598ef86043ffbe5e63c8b5d80cd8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:25.158909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac889d-8ae4-4661-8254-256d8093873b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:25.160595 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac889d-8ae4-4661-8254-256d8093873b admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:25.160878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac889d-8ae4-4661-8254-256d8093873b admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:25.161657 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac889d-8ae4-4661-8254-256d8093873b admin admin] PATH_INFO: `/v3/projects/addc7988230a4022b4462fb9f31afcef/users/e00ff4ac3bad4c1fbb1e9daa9ec09483/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:25.188949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fb4af5f9-fd47-4aca-9407-ddfb63200825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e206598ef86043ffbe5e63c8b5d80cd8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:25.195228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac889d-8ae4-4661-8254-256d8093873b admin admin] RBAC: Authorizing `identity:create_grant(project_id=addc7988230a4022b4462fb9f31afcef, user_id=e00ff4ac3bad4c1fbb1e9daa9ec09483, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:25.199961 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac889d-8ae4-4661-8254-256d8093873b admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:25.205455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:25.222518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 988/1951] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:24 2022] DELETE /identity/v3/users/e206598ef86043ffbe5e63c8b5d80cd8 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:25.224613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 964/1952] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:25 2022] PUT /identity/v3/projects/addc7988230a4022b4462fb9f31afcef/users/e00ff4ac3bad4c1fbb1e9daa9ec09483/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:25.234006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b85c0b-1174-45c8-8248-e6373b6e950d None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:25.234006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b85c0b-1174-45c8-8248-e6373b6e950d None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:25.234006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b85c0b-1174-45c8-8248-e6373b6e950d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:25.237031 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffd807-a4ae-4a72-8344-f895ab8371f7 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:25.237535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffd807-a4ae-4a72-8344-f895ab8371f7 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:25.238921 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffd807-a4ae-4a72-8344-f895ab8371f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:25.268584 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-83b85c0b-1174-45c8-8248-e6373b6e950d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:25.269840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-83b85c0b-1174-45c8-8248-e6373b6e950d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:25.275663 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-f0ffd807-a4ae-4a72-8344-f895ab8371f7 None None] MFA Rules not processed for user `e00ff4ac3bad4c1fbb1e9daa9ec09483`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:25.280074 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f0ffd807-a4ae-4a72-8344-f895ab8371f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:25.387943 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 965/1953] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:25.389297 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffd807-a4ae-4a72-8344-f895ab8371f7 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:25.392380 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f0ffd807-a4ae-4a72-8344-f895ab8371f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:25.402596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 989/1954] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:25.422875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b85c0b-1174-45c8-8248-e6373b6e950d None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:25.425094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-83b85c0b-1174-45c8-8248-e6373b6e950d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:25.433993 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffd807-a4ae-4a72-8344-f895ab8371f7 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:25.441405 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe0c22-92e3-4a84-89a6-0de840cccb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:25.442488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe0c22-92e3-4a84-89a6-0de840cccb74 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:25.442488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe0c22-92e3-4a84-89a6-0de840cccb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:25.442670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe0c22-92e3-4a84-89a6-0de840cccb74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:25.443203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe0c22-92e3-4a84-89a6-0de840cccb74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:25.445434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe0c22-92e3-4a84-89a6-0de840cccb74 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:25.489208 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 966/1955] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:25 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:25.493989 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe0c22-92e3-4a84-89a6-0de840cccb74 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:25.541280 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b85c0b-1174-45c8-8248-e6373b6e950d None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:25.599036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe0c22-92e3-4a84-89a6-0de840cccb74 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:25.605462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0deeb-c2b2-471b-8bb7-ffa369a9f8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:25.606796 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0deeb-c2b2-471b-8bb7-ffa369a9f8ff admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:25.607265 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0deeb-c2b2-471b-8bb7-ffa369a9f8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:25.607916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0deeb-c2b2-471b-8bb7-ffa369a9f8ff admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:25.609330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0deeb-c2b2-471b-8bb7-ffa369a9f8ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:25.611806 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0deeb-c2b2-471b-8bb7-ffa369a9f8ff admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:25.640049 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194c7c7-3a4c-4412-b854-2f6834d06505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:25.640772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194c7c7-3a4c-4412-b854-2f6834d06505 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:25.641147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194c7c7-3a4c-4412-b854-2f6834d06505 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:25.641517 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194c7c7-3a4c-4412-b854-2f6834d06505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:25.642991 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 967/1956] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:25 2022] POST /identity/v3/users => generated 296 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:25.643979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0deeb-c2b2-471b-8bb7-ffa369a9f8ff admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:25.658998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:25.662005 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-faf0deeb-c2b2-471b-8bb7-ffa369a9f8ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:25.663669 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194c7c7-3a4c-4412-b854-2f6834d06505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:25.665680 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194c7c7-3a4c-4412-b854-2f6834d06505 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:25.678090 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 990/1957] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:13:25.678731 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0deeb-c2b2-471b-8bb7-ffa369a9f8ff admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:25.679387 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194c7c7-3a4c-4412-b854-2f6834d06505 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:25.686139 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e93a1e-e56a-4818-ab8c-a95c85dca0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:25.686831 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e93a1e-e56a-4818-ab8c-a95c85dca0a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:25.687111 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e93a1e-e56a-4818-ab8c-a95c85dca0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:25.687387 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e93a1e-e56a-4818-ab8c-a95c85dca0a7 admin admin] PATH_INFO: `/v3/projects/985077d780104111a82dca33cdb43f22` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:25.688597 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e93a1e-e56a-4818-ab8c-a95c85dca0a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=985077d780104111a82dca33cdb43f22)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:25.692881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e93a1e-e56a-4818-ab8c-a95c85dca0a7 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:25.695635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194c7c7-3a4c-4412-b854-2f6834d06505 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:25.705225 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63405a11-e3dc-48d9-b985-7531c2542bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:25.706846 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63405a11-e3dc-48d9-b985-7531c2542bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:25.707245 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63405a11-e3dc-48d9-b985-7531c2542bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:25.707666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63405a11-e3dc-48d9-b985-7531c2542bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:25.709203 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63405a11-e3dc-48d9-b985-7531c2542bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:25.717227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63405a11-e3dc-48d9-b985-7531c2542bb1 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:25.734063 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 991/1958] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:25.739512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63405a11-e3dc-48d9-b985-7531c2542bb1 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:25.761697 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63405a11-e3dc-48d9-b985-7531c2542bb1 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:25.775114 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a3fa9-ec9f-443a-933e-c50c3639944e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:25.775114 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a3fa9-ec9f-443a-933e-c50c3639944e admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:25.775114 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a3fa9-ec9f-443a-933e-c50c3639944e admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:25.775114 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a3fa9-ec9f-443a-933e-c50c3639944e admin admin] PATH_INFO: `/v3/projects/4e8df44acf1249ef9f9ddeb8ef945f0c/users/051ca8afa40f4c06a622328fd77402e8/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:25.792773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a3fa9-ec9f-443a-933e-c50c3639944e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8df44acf1249ef9f9ddeb8ef945f0c, user_id=051ca8afa40f4c06a622328fd77402e8, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:25.795710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a3fa9-ec9f-443a-933e-c50c3639944e admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:25.819010 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-c8e93a1e-e56a-4818-ab8c-a95c85dca0a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '985077d780104111a82dca33cdb43f22', 'request_id': 'req-c8e93a1e-e56a-4818-ab8c-a95c85dca0a7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:25.822322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:25.823922 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-c8e93a1e-e56a-4818-ab8c-a95c85dca0a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '985077d780104111a82dca33cdb43f22', 'request_id': 'req-c8e93a1e-e56a-4818-ab8c-a95c85dca0a7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:25.834517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-c8e93a1e-e56a-4818-ab8c-a95c85dca0a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '985077d780104111a82dca33cdb43f22', 'request_id': 'req-c8e93a1e-e56a-4818-ab8c-a95c85dca0a7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:25.838624 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 992/1959] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:25 2022] PUT /identity/v3/projects/4e8df44acf1249ef9f9ddeb8ef945f0c/users/051ca8afa40f4c06a622328fd77402e8/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:25.842354 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a3fa9-ec9f-443a-933e-c50c3639944e admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:25.857122 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-c8e93a1e-e56a-4818-ab8c-a95c85dca0a7 admin admin] The token cache is being invalidate because project 985077d780104111a82dca33cdb43f22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:25.857449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-c8e93a1e-e56a-4818-ab8c-a95c85dca0a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:25.858499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-c8e93a1e-e56a-4818-ab8c-a95c85dca0a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:25.860599 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 968/1960] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:25 2022] DELETE /identity/v3/projects/985077d780104111a82dca33cdb43f22 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:25.861186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e93a1e-e56a-4818-ab8c-a95c85dca0a7 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:25.862653 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c8e93a1e-e56a-4818-ab8c-a95c85dca0a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:25.948518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a3fa9-ec9f-443a-933e-c50c3639944e admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:25.954122 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73a27a-94ab-44d5-aca4-fe76318d7226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:25.954759 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73a27a-94ab-44d5-aca4-fe76318d7226 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:25.955109 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73a27a-94ab-44d5-aca4-fe76318d7226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:25.955580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73a27a-94ab-44d5-aca4-fe76318d7226 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:25.958733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73a27a-94ab-44d5-aca4-fe76318d7226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:25.960279 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73a27a-94ab-44d5-aca4-fe76318d7226 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:25.975655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 993/1961] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:25.977341 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185a70b-5389-4969-9d8c-bcf841866e9a None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:25.977990 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185a70b-5389-4969-9d8c-bcf841866e9a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:25.978411 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185a70b-5389-4969-9d8c-bcf841866e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:25.987041 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e93a1e-e56a-4818-ab8c-a95c85dca0a7 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:26.006463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dab09-95b2-485d-ae81-3b232373921e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:26.006463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dab09-95b2-485d-ae81-3b232373921e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:26.006463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dab09-95b2-485d-ae81-3b232373921e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:26.006463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dab09-95b2-485d-ae81-3b232373921e admin admin] PATH_INFO: `/v3/users/139e62ed66de4aa2a5874b3907501670` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:26.006945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dab09-95b2-485d-ae81-3b232373921e admin admin] RBAC: Authorizing `identity:delete_user(user_id=139e62ed66de4aa2a5874b3907501670)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:26.010250 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dab09-95b2-485d-ae81-3b232373921e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:26.019672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-d185a70b-5389-4969-9d8c-bcf841866e9a None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:26.024434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d185a70b-5389-4969-9d8c-bcf841866e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:26.054927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 994/1962] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:26.058206 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185a70b-5389-4969-9d8c-bcf841866e9a None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:26.063292 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d185a70b-5389-4969-9d8c-bcf841866e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:26.100148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185a70b-5389-4969-9d8c-bcf841866e9a None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:26.123854 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-227dab09-95b2-485d-ae81-3b232373921e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '139e62ed66de4aa2a5874b3907501670'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:26.125240 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-227dab09-95b2-485d-ae81-3b232373921e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '139e62ed66de4aa2a5874b3907501670'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:26.126210 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-227dab09-95b2-485d-ae81-3b232373921e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '139e62ed66de4aa2a5874b3907501670'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:26.130259 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2577535-5116-49a9-97e3-a30bcac72ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:26.132225 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2577535-5116-49a9-97e3-a30bcac72ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:26.132505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2577535-5116-49a9-97e3-a30bcac72ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:26.134468 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2577535-5116-49a9-97e3-a30bcac72ecd admin admin] PATH_INFO: `/v3/projects/4e8df44acf1249ef9f9ddeb8ef945f0c/users/051ca8afa40f4c06a622328fd77402e8/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:26.139851 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-227dab09-95b2-485d-ae81-3b232373921e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '139e62ed66de4aa2a5874b3907501670'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:26.144703 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2577535-5116-49a9-97e3-a30bcac72ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8df44acf1249ef9f9ddeb8ef945f0c, user_id=051ca8afa40f4c06a622328fd77402e8, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:26.148387 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2577535-5116-49a9-97e3-a30bcac72ecd admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:26.169100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-227dab09-95b2-485d-ae81-3b232373921e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '139e62ed66de4aa2a5874b3907501670'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:26.173076 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 995/1963] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:26 2022] PUT /identity/v3/projects/4e8df44acf1249ef9f9ddeb8ef945f0c/users/051ca8afa40f4c06a622328fd77402e8/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:26.174719 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f2799-f089-473d-a43f-db193a6b4ff2 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:26.175655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f2799-f089-473d-a43f-db193a6b4ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:26.175979 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f2799-f089-473d-a43f-db193a6b4ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:26.194146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 969/1964] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:25 2022] DELETE /identity/v3/users/139e62ed66de4aa2a5874b3907501670 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:26.199513 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d6fa4-b477-4057-98e9-306831396774 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:26.200073 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d6fa4-b477-4057-98e9-306831396774 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:26.200527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d6fa4-b477-4057-98e9-306831396774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:26.208608 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-851f2799-f089-473d-a43f-db193a6b4ff2 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:26.210297 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-851f2799-f089-473d-a43f-db193a6b4ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:26.254272 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-f72d6fa4-b477-4057-98e9-306831396774 None None] MFA Rules not processed for user `051ca8afa40f4c06a622328fd77402e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:26.255925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f72d6fa4-b477-4057-98e9-306831396774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:26.318590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:26.319138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 996/1965] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:26.321554 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75165d7-5536-40a1-a067-8e67e027dac9 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:26.323322 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75165d7-5536-40a1-a067-8e67e027dac9 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:26.323725 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75165d7-5536-40a1-a067-8e67e027dac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:26.350150 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-a75165d7-5536-40a1-a067-8e67e027dac9 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:26.350762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a75165d7-5536-40a1-a067-8e67e027dac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:26.357918 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 970/1966] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:26 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:26.358872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d6fa4-b477-4057-98e9-306831396774 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:26.372297 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 997/1967] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:26.373242 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75165d7-5536-40a1-a067-8e67e027dac9 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:26.374622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a75165d7-5536-40a1-a067-8e67e027dac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:26.382902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d6fa4-b477-4057-98e9-306831396774 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:26.388810 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574a98e-5716-4f95-91cb-864dacbd3b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:26.389424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574a98e-5716-4f95-91cb-864dacbd3b33 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:26.389632 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75165d7-5536-40a1-a067-8e67e027dac9 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:26.389754 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574a98e-5716-4f95-91cb-864dacbd3b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:26.389953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574a98e-5716-4f95-91cb-864dacbd3b33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:26.390614 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574a98e-5716-4f95-91cb-864dacbd3b33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:26.390918 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574a98e-5716-4f95-91cb-864dacbd3b33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:26.392896 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574a98e-5716-4f95-91cb-864dacbd3b33 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:26.397193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28c037-564c-480c-821d-5d9199ebbc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:26.398287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c037-564c-480c-821d-5d9199ebbc7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:26.398717 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c037-564c-480c-821d-5d9199ebbc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:26.399137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c037-564c-480c-821d-5d9199ebbc7c admin admin] PATH_INFO: `/v3/projects/da39cfc285944558a9b45ac8deb96f12` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:26.400698 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28c037-564c-480c-821d-5d9199ebbc7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=da39cfc285944558a9b45ac8deb96f12)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:26.409434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28c037-564c-480c-821d-5d9199ebbc7c admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:26.424271 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 971/1968] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:26.428467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574a98e-5716-4f95-91cb-864dacbd3b33 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:26.430893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2574a98e-5716-4f95-91cb-864dacbd3b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:26.490610 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:26.516582 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-cd28c037-564c-480c-821d-5d9199ebbc7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da39cfc285944558a9b45ac8deb96f12', 'request_id': 'req-cd28c037-564c-480c-821d-5d9199ebbc7c'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:26.526033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-cd28c037-564c-480c-821d-5d9199ebbc7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da39cfc285944558a9b45ac8deb96f12', 'request_id': 'req-cd28c037-564c-480c-821d-5d9199ebbc7c'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:26.540181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-cd28c037-564c-480c-821d-5d9199ebbc7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da39cfc285944558a9b45ac8deb96f12', 'request_id': 'req-cd28c037-564c-480c-821d-5d9199ebbc7c'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:26.546156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574a98e-5716-4f95-91cb-864dacbd3b33 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:26.555371 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-cd28c037-564c-480c-821d-5d9199ebbc7c admin admin] The token cache is being invalidate because project da39cfc285944558a9b45ac8deb96f12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:26.555717 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-cd28c037-564c-480c-821d-5d9199ebbc7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:26.556548 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-cd28c037-564c-480c-821d-5d9199ebbc7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:26.558187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 998/1969] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:26 2022] DELETE /identity/v3/projects/da39cfc285944558a9b45ac8deb96f12 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:26.559072 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28c037-564c-480c-821d-5d9199ebbc7c admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:26.560464 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-cd28c037-564c-480c-821d-5d9199ebbc7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:26.667440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28c037-564c-480c-821d-5d9199ebbc7c admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:26.672115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7314673-3ec5-48bb-830f-4dbb3a14461d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:26.672751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7314673-3ec5-48bb-830f-4dbb3a14461d service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:26.673036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7314673-3ec5-48bb-830f-4dbb3a14461d service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:26.673442 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7314673-3ec5-48bb-830f-4dbb3a14461d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:26.677098 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c7314673-3ec5-48bb-830f-4dbb3a14461d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:26.679156 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75229d9-c76a-41aa-9edb-247f7cc1a465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:26.687505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75229d9-c76a-41aa-9edb-247f7cc1a465 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:26.687505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75229d9-c76a-41aa-9edb-247f7cc1a465 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:26.687505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75229d9-c76a-41aa-9edb-247f7cc1a465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:26.687873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e75229d9-c76a-41aa-9edb-247f7cc1a465 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:26.839336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7314673-3ec5-48bb-830f-4dbb3a14461d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:26.840855 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7314673-3ec5-48bb-830f-4dbb3a14461d service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:26.844317 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75229d9-c76a-41aa-9edb-247f7cc1a465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:26.845874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75229d9-c76a-41aa-9edb-247f7cc1a465 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:26.869619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 999/1970] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:26.870555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75229d9-c76a-41aa-9edb-247f7cc1a465 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:26.871749 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 972/1971] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:26.872768 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7314673-3ec5-48bb-830f-4dbb3a14461d service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:26.873739 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e75229d9-c76a-41aa-9edb-247f7cc1a465 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:26.875999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c7314673-3ec5-48bb-830f-4dbb3a14461d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:26.900339 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7314673-3ec5-48bb-830f-4dbb3a14461d service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:26.901762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75229d9-c76a-41aa-9edb-247f7cc1a465 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:26.920637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34faac0-b74f-4bae-b518-d0e9a2b10f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:26.921050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34faac0-b74f-4bae-b518-d0e9a2b10f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:26.922112 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15642d9-bf97-4bd7-9a48-75b1c9fc1271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:26.922821 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15642d9-bf97-4bd7-9a48-75b1c9fc1271 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:26.923172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15642d9-bf97-4bd7-9a48-75b1c9fc1271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:26.924005 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15642d9-bf97-4bd7-9a48-75b1c9fc1271 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:26.924592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15642d9-bf97-4bd7-9a48-75b1c9fc1271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:26.927248 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15642d9-bf97-4bd7-9a48-75b1c9fc1271 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:26.935332 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34faac0-b74f-4bae-b518-d0e9a2b10f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:26.935764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34faac0-b74f-4bae-b518-d0e9a2b10f84 admin admin] PATH_INFO: `/v3/users/4a91f43c74dc48ada0b8dabf281c5d0f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:26.937239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34faac0-b74f-4bae-b518-d0e9a2b10f84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a91f43c74dc48ada0b8dabf281c5d0f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:26.939373 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34faac0-b74f-4bae-b518-d0e9a2b10f84 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:27.005358 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:27.007516 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1000/1972] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:26 2022] POST /identity/v3/projects => generated 381 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:27.015077 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15642d9-bf97-4bd7-9a48-75b1c9fc1271 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:27.062067 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b34faac0-b74f-4bae-b518-d0e9a2b10f84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a91f43c74dc48ada0b8dabf281c5d0f'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:27.066343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b34faac0-b74f-4bae-b518-d0e9a2b10f84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a91f43c74dc48ada0b8dabf281c5d0f'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:27.067016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b34faac0-b74f-4bae-b518-d0e9a2b10f84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a91f43c74dc48ada0b8dabf281c5d0f'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:27.084566 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b34faac0-b74f-4bae-b518-d0e9a2b10f84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a91f43c74dc48ada0b8dabf281c5d0f'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:27.119778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b34faac0-b74f-4bae-b518-d0e9a2b10f84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a91f43c74dc48ada0b8dabf281c5d0f'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:27.154499 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:27.156204 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 973/1973] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:26 2022] DELETE /identity/v3/users/4a91f43c74dc48ada0b8dabf281c5d0f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:27.228131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15642d9-bf97-4bd7-9a48-75b1c9fc1271 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:27.316599 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86ac42-53c3-4541-9eae-b367b7fb9974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:27.317232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86ac42-53c3-4541-9eae-b367b7fb9974 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:27.317517 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86ac42-53c3-4541-9eae-b367b7fb9974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:27.317820 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86ac42-53c3-4541-9eae-b367b7fb9974 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:27.318462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86ac42-53c3-4541-9eae-b367b7fb9974 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:27.320729 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86ac42-53c3-4541-9eae-b367b7fb9974 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:27.378082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1001/1974] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:27 2022] POST /identity/v3/users => generated 291 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:27.379319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34faac0-b74f-4bae-b518-d0e9a2b10f84 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:27.380862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b34faac0-b74f-4bae-b518-d0e9a2b10f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:27.432588 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34faac0-b74f-4bae-b518-d0e9a2b10f84 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:27.440521 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fbf96f-f4af-4a02-bf32-a2333b4bab42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:27.441708 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbf96f-f4af-4a02-bf32-a2333b4bab42 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:27.442308 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbf96f-f4af-4a02-bf32-a2333b4bab42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:27.442978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbf96f-f4af-4a02-bf32-a2333b4bab42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:27.444142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fbf96f-f4af-4a02-bf32-a2333b4bab42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:27.445901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fbf96f-f4af-4a02-bf32-a2333b4bab42 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:27.473166 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 974/1975] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:27.473714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86ac42-53c3-4541-9eae-b367b7fb9974 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:27.490218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86ac42-53c3-4541-9eae-b367b7fb9974 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:27.496373 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b17a37-197c-456d-9b0b-5c182f042777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:27.497240 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b17a37-197c-456d-9b0b-5c182f042777 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:27.497582 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b17a37-197c-456d-9b0b-5c182f042777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:27.497926 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b17a37-197c-456d-9b0b-5c182f042777 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:27.498647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b17a37-197c-456d-9b0b-5c182f042777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:27.501087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b17a37-197c-456d-9b0b-5c182f042777 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:27.519189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1002/1976] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:27.521140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fbf96f-f4af-4a02-bf32-a2333b4bab42 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:27.535396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fbf96f-f4af-4a02-bf32-a2333b4bab42 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:27.542710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac2404-8f5f-40c4-8a02-25f421e627e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:27.544183 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac2404-8f5f-40c4-8a02-25f421e627e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:27.544725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac2404-8f5f-40c4-8a02-25f421e627e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:27.545238 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac2404-8f5f-40c4-8a02-25f421e627e9 admin admin] PATH_INFO: `/v3/projects/310d40578da34892a86bf9daf0051ff2/users/870b5b5b84464e2f9075502bf4a8f839/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:27.558261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b17a37-197c-456d-9b0b-5c182f042777 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:27.560290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c0b17a37-197c-456d-9b0b-5c182f042777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:27.581330 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b17a37-197c-456d-9b0b-5c182f042777 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:27.592563 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9a83e-fb3e-460c-a97c-d79846f2fba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:27.593504 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9a83e-fb3e-460c-a97c-d79846f2fba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:27.593949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9a83e-fb3e-460c-a97c-d79846f2fba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:27.594072 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac2404-8f5f-40c4-8a02-25f421e627e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=310d40578da34892a86bf9daf0051ff2, user_id=870b5b5b84464e2f9075502bf4a8f839, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:27.594493 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9a83e-fb3e-460c-a97c-d79846f2fba8 admin admin] PATH_INFO: `/v3/projects/21ed2d9c51f44fd09743b5bfc408e240` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:27.595856 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9a83e-fb3e-460c-a97c-d79846f2fba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21ed2d9c51f44fd09743b5bfc408e240)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:27.598126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9a83e-fb3e-460c-a97c-d79846f2fba8 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:27.599347 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac2404-8f5f-40c4-8a02-25f421e627e9 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:27.628345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 975/1977] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:27 2022] PUT /identity/v3/projects/310d40578da34892a86bf9daf0051ff2/users/870b5b5b84464e2f9075502bf4a8f839/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:27.630456 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac2404-8f5f-40c4-8a02-25f421e627e9 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:27.633670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-49ac2404-8f5f-40c4-8a02-25f421e627e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:27.682645 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5db9a83e-fb3e-460c-a97c-d79846f2fba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21ed2d9c51f44fd09743b5bfc408e240', 'request_id': 'req-5db9a83e-fb3e-460c-a97c-d79846f2fba8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:27.690261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5db9a83e-fb3e-460c-a97c-d79846f2fba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21ed2d9c51f44fd09743b5bfc408e240', 'request_id': 'req-5db9a83e-fb3e-460c-a97c-d79846f2fba8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:27.703358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5db9a83e-fb3e-460c-a97c-d79846f2fba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21ed2d9c51f44fd09743b5bfc408e240', 'request_id': 'req-5db9a83e-fb3e-460c-a97c-d79846f2fba8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:27.720548 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5db9a83e-fb3e-460c-a97c-d79846f2fba8 admin admin] The token cache is being invalidate because project 21ed2d9c51f44fd09743b5bfc408e240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:27.720862 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5db9a83e-fb3e-460c-a97c-d79846f2fba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:27.721541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5db9a83e-fb3e-460c-a97c-d79846f2fba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:27.727462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1003/1978] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:27 2022] DELETE /identity/v3/projects/21ed2d9c51f44fd09743b5bfc408e240 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:27.729270 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1cdc7-c1cf-4d4d-9f27-f0c8fc7264d9 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:27.729553 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1cdc7-c1cf-4d4d-9f27-f0c8fc7264d9 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:27.729828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1cdc7-c1cf-4d4d-9f27-f0c8fc7264d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:27.749881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac2404-8f5f-40c4-8a02-25f421e627e9 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:27.762155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-04d1cdc7-c1cf-4d4d-9f27-f0c8fc7264d9 None None] MFA Rules not processed for user `870b5b5b84464e2f9075502bf4a8f839`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:27.763698 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-04d1cdc7-c1cf-4d4d-9f27-f0c8fc7264d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:27.816566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:27.820231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798eee6c-de65-4e30-a230-25e5768b0bad tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '051ca8afa40f4c06a622328fd77402e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e8df44acf1249ef9f9ddeb8ef945f0c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:27.821000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798eee6c-de65-4e30-a230-25e5768b0bad tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:27.821290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798eee6c-de65-4e30-a230-25e5768b0bad tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:27.821808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798eee6c-de65-4e30-a230-25e5768b0bad tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:27.822476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798eee6c-de65-4e30-a230-25e5768b0bad tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:27.822971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798eee6c-de65-4e30-a230-25e5768b0bad tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:27.824864 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798eee6c-de65-4e30-a230-25e5768b0bad tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:27.838491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 976/1979] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:27.839195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798eee6c-de65-4e30-a230-25e5768b0bad tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:27.841075 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-798eee6c-de65-4e30-a230-25e5768b0bad tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:27.849383 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1004/1980] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:27.851016 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce752cca-51ec-45a4-9c30-5632223e7601 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:27.851547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce752cca-51ec-45a4-9c30-5632223e7601 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:27.852169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce752cca-51ec-45a4-9c30-5632223e7601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:27.854546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:27.875739 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-ce752cca-51ec-45a4-9c30-5632223e7601 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:27.876961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ce752cca-51ec-45a4-9c30-5632223e7601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:27.921893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798eee6c-de65-4e30-a230-25e5768b0bad tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:27.928426 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be5137-2d4c-4852-832a-c39aadfe8d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:27.929448 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be5137-2d4c-4852-832a-c39aadfe8d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:27.929736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be5137-2d4c-4852-832a-c39aadfe8d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:27.930015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be5137-2d4c-4852-832a-c39aadfe8d29 admin admin] PATH_INFO: `/v3/projects/d9ed83b36f0d4cd5b158c197aa2b10a8` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:27.932077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be5137-2d4c-4852-832a-c39aadfe8d29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ed83b36f0d4cd5b158c197aa2b10a8)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:27.934894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be5137-2d4c-4852-832a-c39aadfe8d29 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:27.953717 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1005/1981] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:27.953717 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce752cca-51ec-45a4-9c30-5632223e7601 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:27.953717 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ce752cca-51ec-45a4-9c30-5632223e7601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:27.963736 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce752cca-51ec-45a4-9c30-5632223e7601 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:27.969038 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3db314-8ac9-48ea-8be6-a51e110b3dc1 tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '051ca8afa40f4c06a622328fd77402e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e8df44acf1249ef9f9ddeb8ef945f0c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:27.969766 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3db314-8ac9-48ea-8be6-a51e110b3dc1 tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:27.970139 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3db314-8ac9-48ea-8be6-a51e110b3dc1 tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:27.970668 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3db314-8ac9-48ea-8be6-a51e110b3dc1 tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:27.971324 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3db314-8ac9-48ea-8be6-a51e110b3dc1 tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:27.973545 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3db314-8ac9-48ea-8be6-a51e110b3dc1 tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:28.017445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1006/1982] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:27 2022] POST /identity/v3/projects => generated 354 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:28.017445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3db314-8ac9-48ea-8be6-a51e110b3dc1 tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:28.017445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4e3db314-8ac9-48ea-8be6-a51e110b3dc1 tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:28.031704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-20be5137-2d4c-4852-832a-c39aadfe8d29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ed83b36f0d4cd5b158c197aa2b10a8', 'request_id': 'req-20be5137-2d4c-4852-832a-c39aadfe8d29'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:28.037637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-20be5137-2d4c-4852-832a-c39aadfe8d29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ed83b36f0d4cd5b158c197aa2b10a8', 'request_id': 'req-20be5137-2d4c-4852-832a-c39aadfe8d29'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:28.048900 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-20be5137-2d4c-4852-832a-c39aadfe8d29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ed83b36f0d4cd5b158c197aa2b10a8', 'request_id': 'req-20be5137-2d4c-4852-832a-c39aadfe8d29'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:28.060972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-20be5137-2d4c-4852-832a-c39aadfe8d29 admin admin] The token cache is being invalidate because project d9ed83b36f0d4cd5b158c197aa2b10a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:28.061262 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-20be5137-2d4c-4852-832a-c39aadfe8d29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:28.061960 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-20be5137-2d4c-4852-832a-c39aadfe8d29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:28.063550 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 977/1983] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:27 2022] DELETE /identity/v3/projects/d9ed83b36f0d4cd5b158c197aa2b10a8 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:28.066931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dabb3-f894-4735-bca3-949c3601f1da None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:28.066931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dabb3-f894-4735-bca3-949c3601f1da None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:28.066931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dabb3-f894-4735-bca3-949c3601f1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:28.091559 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-a67dabb3-f894-4735-bca3-949c3601f1da None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:28.092751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-a67dabb3-f894-4735-bca3-949c3601f1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:28.094569 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3db314-8ac9-48ea-8be6-a51e110b3dc1 tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:28.099168 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5063d05-97b4-4849-bd89-17f5f9ee608f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:28.099830 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5063d05-97b4-4849-bd89-17f5f9ee608f admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:28.100110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5063d05-97b4-4849-bd89-17f5f9ee608f admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:28.100399 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5063d05-97b4-4849-bd89-17f5f9ee608f admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:28.101011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5063d05-97b4-4849-bd89-17f5f9ee608f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:28.103117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5063d05-97b4-4849-bd89-17f5f9ee608f admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:28.108613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 978/1984] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:28.109713 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dabb3-f894-4735-bca3-949c3601f1da None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:28.111692 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-a67dabb3-f894-4735-bca3-949c3601f1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:28.145099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1007/1985] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:28 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:28.145991 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5063d05-97b4-4849-bd89-17f5f9ee608f admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:28.149162 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d5063d05-97b4-4849-bd89-17f5f9ee608f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:28.190541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dabb3-f894-4735-bca3-949c3601f1da None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:28.198106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:28.217480 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5063d05-97b4-4849-bd89-17f5f9ee608f admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:28.222578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c32bd-30d5-430b-adf5-16a5eeae8d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:28.222832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c32bd-30d5-430b-adf5-16a5eeae8d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:28.223111 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c32bd-30d5-430b-adf5-16a5eeae8d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:28.223388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c32bd-30d5-430b-adf5-16a5eeae8d22 admin admin] PATH_INFO: `/v3/users/067f97a0734d428c99f3d041c55f5a6a` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:28.224579 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c32bd-30d5-430b-adf5-16a5eeae8d22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=067f97a0734d428c99f3d041c55f5a6a)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:28.226581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c32bd-30d5-430b-adf5-16a5eeae8d22 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:28.234602 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:28.256657 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154fd92f-40f2-4d6b-b8ff-119cc7956aa8 tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '051ca8afa40f4c06a622328fd77402e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e8df44acf1249ef9f9ddeb8ef945f0c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:28.257282 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fd92f-40f2-4d6b-b8ff-119cc7956aa8 tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:28.257593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fd92f-40f2-4d6b-b8ff-119cc7956aa8 tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:28.257870 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fd92f-40f2-4d6b-b8ff-119cc7956aa8 tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:28.258536 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154fd92f-40f2-4d6b-b8ff-119cc7956aa8 tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:28.258844 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154fd92f-40f2-4d6b-b8ff-119cc7956aa8 tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:28.260934 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154fd92f-40f2-4d6b-b8ff-119cc7956aa8 tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:28.275130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 979/1986] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:28.276056 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154fd92f-40f2-4d6b-b8ff-119cc7956aa8 tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:28.288343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154fd92f-40f2-4d6b-b8ff-119cc7956aa8 tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:28.293481 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055081fd-5619-4352-b15f-0e91b31bdeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:28.294949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055081fd-5619-4352-b15f-0e91b31bdeb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:28.295383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055081fd-5619-4352-b15f-0e91b31bdeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:28.295862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055081fd-5619-4352-b15f-0e91b31bdeb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:28.296562 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055081fd-5619-4352-b15f-0e91b31bdeb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:28.296870 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055081fd-5619-4352-b15f-0e91b31bdeb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:28.299309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055081fd-5619-4352-b15f-0e91b31bdeb2 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:28.300754 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c94c32bd-30d5-430b-adf5-16a5eeae8d22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '067f97a0734d428c99f3d041c55f5a6a'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:28.301598 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c94c32bd-30d5-430b-adf5-16a5eeae8d22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '067f97a0734d428c99f3d041c55f5a6a'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:28.302671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c94c32bd-30d5-430b-adf5-16a5eeae8d22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '067f97a0734d428c99f3d041c55f5a6a'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:28.313229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c94c32bd-30d5-430b-adf5-16a5eeae8d22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '067f97a0734d428c99f3d041c55f5a6a'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:28.321547 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 980/1987] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:28.321547 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055081fd-5619-4352-b15f-0e91b31bdeb2 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:28.321547 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-055081fd-5619-4352-b15f-0e91b31bdeb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:28.336282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c94c32bd-30d5-430b-adf5-16a5eeae8d22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '067f97a0734d428c99f3d041c55f5a6a'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:28.343447 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055081fd-5619-4352-b15f-0e91b31bdeb2 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:28.348275 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f9739-959b-48e9-933a-fdcf43b9ad59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:28.348878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f9739-959b-48e9-933a-fdcf43b9ad59 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:28.349154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f9739-959b-48e9-933a-fdcf43b9ad59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:28.349426 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f9739-959b-48e9-933a-fdcf43b9ad59 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:28.350052 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f9739-959b-48e9-933a-fdcf43b9ad59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:28.350857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1008/1988] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:28 2022] DELETE /identity/v3/users/067f97a0734d428c99f3d041c55f5a6a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:28.351755 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c32bd-30d5-430b-adf5-16a5eeae8d22 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:28.352679 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f9739-959b-48e9-933a-fdcf43b9ad59 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:28.353327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c94c32bd-30d5-430b-adf5-16a5eeae8d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:28.380685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 981/1989] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:28 2022] POST /identity/v3/users => generated 297 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:28.381527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f9739-959b-48e9-933a-fdcf43b9ad59 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:28.383095 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ca2f9739-959b-48e9-933a-fdcf43b9ad59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:28.396753 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f9739-959b-48e9-933a-fdcf43b9ad59 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:28.437386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c32bd-30d5-430b-adf5-16a5eeae8d22 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:28.442500 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79497da6-1f77-42b0-a746-64d553381541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:28.443231 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79497da6-1f77-42b0-a746-64d553381541 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:28.443718 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79497da6-1f77-42b0-a746-64d553381541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:28.444060 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79497da6-1f77-42b0-a746-64d553381541 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:28.444843 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79497da6-1f77-42b0-a746-64d553381541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:28.446931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79497da6-1f77-42b0-a746-64d553381541 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:28.466224 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c4ee1-36ea-434f-9424-ea730ad1b41a tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '051ca8afa40f4c06a622328fd77402e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e8df44acf1249ef9f9ddeb8ef945f0c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:28.466745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c4ee1-36ea-434f-9424-ea730ad1b41a tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:28.467364 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c4ee1-36ea-434f-9424-ea730ad1b41a tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:28.467470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c4ee1-36ea-434f-9424-ea730ad1b41a tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] PATH_INFO: `/v3/projects/1a88bb6061d14bd0b709c170c0a47dad` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:28.468953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c4ee1-36ea-434f-9424-ea730ad1b41a tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1a88bb6061d14bd0b709c170c0a47dad)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:28.470863 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c4ee1-36ea-434f-9424-ea730ad1b41a tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:28.483523 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1009/1990] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:28 2022] POST /identity/v3/projects => generated 377 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:28.483523 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79497da6-1f77-42b0-a746-64d553381541 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:28.544050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:28.552779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b22c4ee1-36ea-434f-9424-ea730ad1b41a tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a88bb6061d14bd0b709c170c0a47dad', 'request_id': 'req-b22c4ee1-36ea-434f-9424-ea730ad1b41a'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:28.566541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b22c4ee1-36ea-434f-9424-ea730ad1b41a tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a88bb6061d14bd0b709c170c0a47dad', 'request_id': 'req-b22c4ee1-36ea-434f-9424-ea730ad1b41a'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:28.568396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79497da6-1f77-42b0-a746-64d553381541 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:28.577370 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b22c4ee1-36ea-434f-9424-ea730ad1b41a tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a88bb6061d14bd0b709c170c0a47dad', 'request_id': 'req-b22c4ee1-36ea-434f-9424-ea730ad1b41a'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:28.593674 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b22c4ee1-36ea-434f-9424-ea730ad1b41a tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] The token cache is being invalidate because project 1a88bb6061d14bd0b709c170c0a47dad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:28.593971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b22c4ee1-36ea-434f-9424-ea730ad1b41a tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:28.594662 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b22c4ee1-36ea-434f-9424-ea730ad1b41a tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:28.596547 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 982/1991] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:28 2022] DELETE /identity/v3/projects/1a88bb6061d14bd0b709c170c0a47dad => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:28.597314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c4ee1-36ea-434f-9424-ea730ad1b41a tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:28.614800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b22c4ee1-36ea-434f-9424-ea730ad1b41a tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:28.660187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6389f-457b-4422-8293-d505ea1ed02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:28.662816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6389f-457b-4422-8293-d505ea1ed02f admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:28.663639 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6389f-457b-4422-8293-d505ea1ed02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:28.664090 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6389f-457b-4422-8293-d505ea1ed02f admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:28.665147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6389f-457b-4422-8293-d505ea1ed02f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:28.669152 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:28.669998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6389f-457b-4422-8293-d505ea1ed02f admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:28.691004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1010/1992] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:28.695535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6389f-457b-4422-8293-d505ea1ed02f admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:28.698925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-f1b6389f-457b-4422-8293-d505ea1ed02f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:28.702932 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c4ee1-36ea-434f-9424-ea730ad1b41a tempest-VolumeQuotaClassesTest-774401018 tempest-VolumeQuotaClassesTest-774401018-project-admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:28.709901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05128f65-8e44-4da8-9ae0-8ad150078f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:28.710541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05128f65-8e44-4da8-9ae0-8ad150078f55 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:28.710944 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05128f65-8e44-4da8-9ae0-8ad150078f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:28.711262 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05128f65-8e44-4da8-9ae0-8ad150078f55 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:28.712003 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05128f65-8e44-4da8-9ae0-8ad150078f55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:28.714731 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05128f65-8e44-4da8-9ae0-8ad150078f55 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:28.722793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6389f-457b-4422-8293-d505ea1ed02f admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:28.732016 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6458e8e-4809-4ead-9af9-2105cb9d7665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:28.732936 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6458e8e-4809-4ead-9af9-2105cb9d7665 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:28.733263 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6458e8e-4809-4ead-9af9-2105cb9d7665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:28.733657 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6458e8e-4809-4ead-9af9-2105cb9d7665 admin admin] PATH_INFO: `/v3/projects/e7bf44a6062d427f8ac5c97228702755/users/a006a62bed3a459a9a280b8fabde4f8e/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:28.752455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 983/1993] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:28 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:28.755947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05128f65-8e44-4da8-9ae0-8ad150078f55 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:28.761032 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6458e8e-4809-4ead-9af9-2105cb9d7665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7bf44a6062d427f8ac5c97228702755, user_id=a006a62bed3a459a9a280b8fabde4f8e, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:28.766353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6458e8e-4809-4ead-9af9-2105cb9d7665 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:28.771209 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05128f65-8e44-4da8-9ae0-8ad150078f55 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:28.786576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949af47b-c7fb-48be-bb80-e745f710d5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:28.788830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949af47b-c7fb-48be-bb80-e745f710d5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:28.788830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949af47b-c7fb-48be-bb80-e745f710d5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:28.788830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949af47b-c7fb-48be-bb80-e745f710d5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:28.789592 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949af47b-c7fb-48be-bb80-e745f710d5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:28.791677 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949af47b-c7fb-48be-bb80-e745f710d5d4 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:28.797024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1011/1994] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:28 2022] PUT /identity/v3/projects/e7bf44a6062d427f8ac5c97228702755/users/a006a62bed3a459a9a280b8fabde4f8e/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:28.803716 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6458e8e-4809-4ead-9af9-2105cb9d7665 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:28.803867 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 984/1995] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:28.808322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949af47b-c7fb-48be-bb80-e745f710d5d4 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:28.825242 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949af47b-c7fb-48be-bb80-e745f710d5d4 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:28.893222 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6458e8e-4809-4ead-9af9-2105cb9d7665 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:28.900965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa692c3-dc10-4ea9-97ed-da379ccfb489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:28.902095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa692c3-dc10-4ea9-97ed-da379ccfb489 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:28.902255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa692c3-dc10-4ea9-97ed-da379ccfb489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:28.902999 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa692c3-dc10-4ea9-97ed-da379ccfb489 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:28.903691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa692c3-dc10-4ea9-97ed-da379ccfb489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:28.905437 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa692c3-dc10-4ea9-97ed-da379ccfb489 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:28.907853 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697788c-19f8-4438-bb7c-437def8eef91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:28.908691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697788c-19f8-4438-bb7c-437def8eef91 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:28.909041 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697788c-19f8-4438-bb7c-437def8eef91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:28.909386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697788c-19f8-4438-bb7c-437def8eef91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:28.910277 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697788c-19f8-4438-bb7c-437def8eef91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:28.911960 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697788c-19f8-4438-bb7c-437def8eef91 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:28.923642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 985/1996] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:28.924645 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697788c-19f8-4438-bb7c-437def8eef91 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:28.926620 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c697788c-19f8-4438-bb7c-437def8eef91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:28.940523 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1012/1997] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:28.940523 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa692c3-dc10-4ea9-97ed-da379ccfb489 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:28.943507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa692c3-dc10-4ea9-97ed-da379ccfb489 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:28.943984 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697788c-19f8-4438-bb7c-437def8eef91 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:28.948767 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046a0c1-6659-4212-8734-8ffe97075532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:28.949504 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046a0c1-6659-4212-8734-8ffe97075532 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:28.949809 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046a0c1-6659-4212-8734-8ffe97075532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:28.949925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7842d-cbdb-45ae-b4c0-00af2cffb4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:28.950087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046a0c1-6659-4212-8734-8ffe97075532 admin admin] PATH_INFO: `/v3/projects/59540531d83242cb80c880f2341cfbbc/users/599ff9e636ee4a8d976208ca0854bda7/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:28.950601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7842d-cbdb-45ae-b4c0-00af2cffb4c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:28.950951 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7842d-cbdb-45ae-b4c0-00af2cffb4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:28.951314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7842d-cbdb-45ae-b4c0-00af2cffb4c9 admin admin] PATH_INFO: `/v3/users/e00ff4ac3bad4c1fbb1e9daa9ec09483` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:28.952593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f7842d-cbdb-45ae-b4c0-00af2cffb4c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e00ff4ac3bad4c1fbb1e9daa9ec09483)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:28.954546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f7842d-cbdb-45ae-b4c0-00af2cffb4c9 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:28.966902 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6046a0c1-6659-4212-8734-8ffe97075532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59540531d83242cb80c880f2341cfbbc, user_id=599ff9e636ee4a8d976208ca0854bda7, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:28.970506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6046a0c1-6659-4212-8734-8ffe97075532 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:28.992250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1013/1998] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:28 2022] PUT /identity/v3/projects/59540531d83242cb80c880f2341cfbbc/users/599ff9e636ee4a8d976208ca0854bda7/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:29.001481 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046a0c1-6659-4212-8734-8ffe97075532 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:29.030687 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-98f7842d-cbdb-45ae-b4c0-00af2cffb4c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e00ff4ac3bad4c1fbb1e9daa9ec09483'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:29.031393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-98f7842d-cbdb-45ae-b4c0-00af2cffb4c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e00ff4ac3bad4c1fbb1e9daa9ec09483'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:29.032046 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-98f7842d-cbdb-45ae-b4c0-00af2cffb4c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e00ff4ac3bad4c1fbb1e9daa9ec09483'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:29.039061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-98f7842d-cbdb-45ae-b4c0-00af2cffb4c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e00ff4ac3bad4c1fbb1e9daa9ec09483'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:29.042293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:29.053547 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-98f7842d-cbdb-45ae-b4c0-00af2cffb4c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e00ff4ac3bad4c1fbb1e9daa9ec09483'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:29.068274 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 986/1999] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:28 2022] DELETE /identity/v3/users/e00ff4ac3bad4c1fbb1e9daa9ec09483 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:29.069477 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7842d-cbdb-45ae-b4c0-00af2cffb4c9 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:29.071077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-98f7842d-cbdb-45ae-b4c0-00af2cffb4c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:29.074684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046a0c1-6659-4212-8734-8ffe97075532 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:29.097115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:29.135760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75e8f4-4f41-4d9a-9820-742ed84ca1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:29.136456 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75e8f4-4f41-4d9a-9820-742ed84ca1d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:29.136748 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75e8f4-4f41-4d9a-9820-742ed84ca1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:29.137026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75e8f4-4f41-4d9a-9820-742ed84ca1d7 admin admin] PATH_INFO: `/v3/projects/e7bf44a6062d427f8ac5c97228702755/users/a006a62bed3a459a9a280b8fabde4f8e/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:29.138680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7842d-cbdb-45ae-b4c0-00af2cffb4c9 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:29.141149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed75e8f4-4f41-4d9a-9820-742ed84ca1d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7bf44a6062d427f8ac5c97228702755, user_id=a006a62bed3a459a9a280b8fabde4f8e, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:29.144205 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b176c-008f-4367-8e9a-93728fb2e2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:29.144864 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b176c-008f-4367-8e9a-93728fb2e2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:29.145144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b176c-008f-4367-8e9a-93728fb2e2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:29.145420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b176c-008f-4367-8e9a-93728fb2e2b2 admin admin] PATH_INFO: `/v3/projects/929d0e35319f45babbf281bfbfb2c66f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:29.145591 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed75e8f4-4f41-4d9a-9820-742ed84ca1d7 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:29.146610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b176c-008f-4367-8e9a-93728fb2e2b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=929d0e35319f45babbf281bfbfb2c66f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:29.148818 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b176c-008f-4367-8e9a-93728fb2e2b2 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:29.157445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1014/2000] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:28 2022] PUT /identity/v3/projects/e7bf44a6062d427f8ac5c97228702755/users/a006a62bed3a459a9a280b8fabde4f8e/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:29.158711 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4da987-6c2b-472e-87cb-25ffebefcf5d None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:29.158985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4da987-6c2b-472e-87cb-25ffebefcf5d None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:29.159258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4da987-6c2b-472e-87cb-25ffebefcf5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:29.188958 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-6d4da987-6c2b-472e-87cb-25ffebefcf5d None None] MFA Rules not processed for user `599ff9e636ee4a8d976208ca0854bda7`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:29.190214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6d4da987-6c2b-472e-87cb-25ffebefcf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:29.199021 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-622b176c-008f-4367-8e9a-93728fb2e2b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '929d0e35319f45babbf281bfbfb2c66f', 'request_id': 'req-622b176c-008f-4367-8e9a-93728fb2e2b2'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:29.204488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-622b176c-008f-4367-8e9a-93728fb2e2b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '929d0e35319f45babbf281bfbfb2c66f', 'request_id': 'req-622b176c-008f-4367-8e9a-93728fb2e2b2'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:29.213377 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-622b176c-008f-4367-8e9a-93728fb2e2b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '929d0e35319f45babbf281bfbfb2c66f', 'request_id': 'req-622b176c-008f-4367-8e9a-93728fb2e2b2'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:29.224861 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-622b176c-008f-4367-8e9a-93728fb2e2b2 admin admin] The token cache is being invalidate because project 929d0e35319f45babbf281bfbfb2c66f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:29.225149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-622b176c-008f-4367-8e9a-93728fb2e2b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:29.225796 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-622b176c-008f-4367-8e9a-93728fb2e2b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:29.227345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 987/2001] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:29 2022] DELETE /identity/v3/projects/929d0e35319f45babbf281bfbfb2c66f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:29.231901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b0f6e-67ba-4677-8ad3-098223bcbd9e None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:29.231901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b0f6e-67ba-4677-8ad3-098223bcbd9e None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:29.231901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b0f6e-67ba-4677-8ad3-098223bcbd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:29.248094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-9a0b0f6e-67ba-4677-8ad3-098223bcbd9e None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:29.249302 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9a0b0f6e-67ba-4677-8ad3-098223bcbd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:29.259632 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1015/2002] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:29.264437 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849ff52-d7f4-4a32-ae2e-7b1a3c2aea0e None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:29.264437 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849ff52-d7f4-4a32-ae2e-7b1a3c2aea0e None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:29.264437 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849ff52-d7f4-4a32-ae2e-7b1a3c2aea0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:29.290216 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-d849ff52-d7f4-4a32-ae2e-7b1a3c2aea0e None None] MFA Rules not processed for user `a006a62bed3a459a9a280b8fabde4f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:29.291529 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d849ff52-d7f4-4a32-ae2e-7b1a3c2aea0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:29.310681 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 988/2003] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:29.312185 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3de4f-0208-4988-912e-9739ff3314a6 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:29.312468 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3de4f-0208-4988-912e-9739ff3314a6 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:29.312743 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3de4f-0208-4988-912e-9739ff3314a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:29.333734 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-01e3de4f-0208-4988-912e-9739ff3314a6 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:29.334881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-01e3de4f-0208-4988-912e-9739ff3314a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:29.344403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 989/2004] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:29.345197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3de4f-0208-4988-912e-9739ff3314a6 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:29.346469 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-01e3de4f-0208-4988-912e-9739ff3314a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:29.363827 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:29.379403 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1016/2005] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:29.379403 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849ff52-d7f4-4a32-ae2e-7b1a3c2aea0e None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:29.379403 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d849ff52-d7f4-4a32-ae2e-7b1a3c2aea0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:29.402843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:29.409507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3de4f-0208-4988-912e-9739ff3314a6 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:29.414551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494b6fc-2dfb-4465-8669-579abae0b1d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:29.415217 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494b6fc-2dfb-4465-8669-579abae0b1d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:29.415631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494b6fc-2dfb-4465-8669-579abae0b1d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:29.416156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494b6fc-2dfb-4465-8669-579abae0b1d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:29.424964 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494b6fc-2dfb-4465-8669-579abae0b1d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:29.426486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494b6fc-2dfb-4465-8669-579abae0b1d3 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:29.435824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 990/2006] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:29.436546 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849ff52-d7f4-4a32-ae2e-7b1a3c2aea0e None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:29.437627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c6d4f-bb1c-4d13-849d-2a6ee1e4c49d None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:29.438201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c6d4f-bb1c-4d13-849d-2a6ee1e4c49d None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:29.438566 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c6d4f-bb1c-4d13-849d-2a6ee1e4c49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:29.440271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ff871-4e2a-41ab-85d2-425f3b82fd64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:29.440998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ff871-4e2a-41ab-85d2-425f3b82fd64 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:29.441281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ff871-4e2a-41ab-85d2-425f3b82fd64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:29.441553 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ff871-4e2a-41ab-85d2-425f3b82fd64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:29.453198 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ff871-4e2a-41ab-85d2-425f3b82fd64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:29.454977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ff871-4e2a-41ab-85d2-425f3b82fd64 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:29.458922 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-3c7c6d4f-bb1c-4d13-849d-2a6ee1e4c49d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:29.460159 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3c7c6d4f-bb1c-4d13-849d-2a6ee1e4c49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:29.466101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1017/2007] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:29.466932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ff871-4e2a-41ab-85d2-425f3b82fd64 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:29.468300 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-cd7ff871-4e2a-41ab-85d2-425f3b82fd64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:29.469016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 991/2008] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:29.472947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c6d4f-bb1c-4d13-849d-2a6ee1e4c49d None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:29.480273 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ff871-4e2a-41ab-85d2-425f3b82fd64 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:29.484214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613865fe-4e52-472a-9706-dff272dfcca5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:29.484686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c6d4f-bb1c-4d13-849d-2a6ee1e4c49d None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:29.484805 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613865fe-4e52-472a-9706-dff272dfcca5 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:29.485085 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613865fe-4e52-472a-9706-dff272dfcca5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:29.485360 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613865fe-4e52-472a-9706-dff272dfcca5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:29.490188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c88b4-6149-4027-accc-4abc1e33a3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:29.490667 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c88b4-6149-4027-accc-4abc1e33a3ad admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:29.490948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c88b4-6149-4027-accc-4abc1e33a3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:29.491395 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c88b4-6149-4027-accc-4abc1e33a3ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:29.492305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c88b4-6149-4027-accc-4abc1e33a3ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:29.492625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c88b4-6149-4027-accc-4abc1e33a3ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:29.495312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c88b4-6149-4027-accc-4abc1e33a3ad admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:29.495673 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613865fe-4e52-472a-9706-dff272dfcca5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:29.497432 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613865fe-4e52-472a-9706-dff272dfcca5 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:29.508239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1018/2009] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:29.512685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 992/2010] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:13:29.516761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613865fe-4e52-472a-9706-dff272dfcca5 service cinder] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:29.530942 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613865fe-4e52-472a-9706-dff272dfcca5 service cinder] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:29.538025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b3261-b49a-494a-9f75-11d3e00a0945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:29.538725 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b3261-b49a-494a-9f75-11d3e00a0945 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:29.539982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b3261-b49a-494a-9f75-11d3e00a0945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:29.539982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b3261-b49a-494a-9f75-11d3e00a0945 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:29.540189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b3261-b49a-494a-9f75-11d3e00a0945 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:29.542054 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b3261-b49a-494a-9f75-11d3e00a0945 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:29.584815 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1019/2011] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:29 2022] POST /identity/v3/projects => generated 389 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:29.587625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c88b4-6149-4027-accc-4abc1e33a3ad admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:29.602280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c88b4-6149-4027-accc-4abc1e33a3ad admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:29.697628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd0b0f-dd83-4857-82d0-a9d0cfd1354c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:29.699183 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd0b0f-dd83-4857-82d0-a9d0cfd1354c admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:29.700216 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd0b0f-dd83-4857-82d0-a9d0cfd1354c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:29.700679 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd0b0f-dd83-4857-82d0-a9d0cfd1354c admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:29.701544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd0b0f-dd83-4857-82d0-a9d0cfd1354c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:29.703904 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd0b0f-dd83-4857-82d0-a9d0cfd1354c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:29.736564 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 993/2012] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:29 2022] POST /identity/v3/users => generated 295 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:29.738756 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b3261-b49a-494a-9f75-11d3e00a0945 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:29.761303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b3261-b49a-494a-9f75-11d3e00a0945 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:29.773326 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4432c-a2c4-4d79-8266-decf40f85804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:29.773979 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4432c-a2c4-4d79-8266-decf40f85804 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:29.775332 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4432c-a2c4-4d79-8266-decf40f85804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:29.775760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4432c-a2c4-4d79-8266-decf40f85804 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:29.776519 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4432c-a2c4-4d79-8266-decf40f85804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:29.778153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4432c-a2c4-4d79-8266-decf40f85804 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:29.793135 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1020/2013] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:29.797863 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd0b0f-dd83-4857-82d0-a9d0cfd1354c admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:29.828702 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd0b0f-dd83-4857-82d0-a9d0cfd1354c admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:29.842900 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dba1d8-eb84-478d-befd-44075603b21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:29.843859 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dba1d8-eb84-478d-befd-44075603b21a admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:29.844812 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dba1d8-eb84-478d-befd-44075603b21a admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:29.845289 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dba1d8-eb84-478d-befd-44075603b21a admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:29.846514 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dba1d8-eb84-478d-befd-44075603b21a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:29.850159 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dba1d8-eb84-478d-befd-44075603b21a admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:29.872615 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 994/2014] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:29.873799 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4432c-a2c4-4d79-8266-decf40f85804 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:29.889748 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4432c-a2c4-4d79-8266-decf40f85804 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:29.900691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a601628-3f9f-4d79-94aa-ae67c413ee26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:29.901666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a601628-3f9f-4d79-94aa-ae67c413ee26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:29.903368 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a601628-3f9f-4d79-94aa-ae67c413ee26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:29.903368 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a601628-3f9f-4d79-94aa-ae67c413ee26 admin admin] PATH_INFO: `/v3/projects/767bd9c6402e4ef7bd283c56eb8b6c98/users/4ce088523a9e4e64994392f1d41e2545/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:29.936785 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dba1d8-eb84-478d-befd-44075603b21a admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:29.937064 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a601628-3f9f-4d79-94aa-ae67c413ee26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=767bd9c6402e4ef7bd283c56eb8b6c98, user_id=4ce088523a9e4e64994392f1d41e2545, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:29.938486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-76dba1d8-eb84-478d-befd-44075603b21a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:29.941228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a601628-3f9f-4d79-94aa-ae67c413ee26 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:29.959764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dba1d8-eb84-478d-befd-44075603b21a admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:29.979794 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1021/2015] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:29 2022] PUT /identity/v3/projects/767bd9c6402e4ef7bd283c56eb8b6c98/users/4ce088523a9e4e64994392f1d41e2545/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:29.988548 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca379bf-e8d3-41db-85ad-f45fda91f7ae None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:29.988830 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca379bf-e8d3-41db-85ad-f45fda91f7ae None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:29.989108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca379bf-e8d3-41db-85ad-f45fda91f7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:30.035936 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-6ca379bf-e8d3-41db-85ad-f45fda91f7ae None None] MFA Rules not processed for user `4ce088523a9e4e64994392f1d41e2545`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:30.037509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6ca379bf-e8d3-41db-85ad-f45fda91f7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:30.084188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9993abe-b631-4f62-ad45-9e34d83dd414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:30.084859 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9993abe-b631-4f62-ad45-9e34d83dd414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:30.085143 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9993abe-b631-4f62-ad45-9e34d83dd414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:30.085416 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9993abe-b631-4f62-ad45-9e34d83dd414 admin admin] PATH_INFO: `/v3/projects/addc7988230a4022b4462fb9f31afcef` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:30.086722 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9993abe-b631-4f62-ad45-9e34d83dd414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=addc7988230a4022b4462fb9f31afcef)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:30.089409 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9993abe-b631-4f62-ad45-9e34d83dd414 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:30.117943 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1022/2016] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:30.127447 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca379bf-e8d3-41db-85ad-f45fda91f7ae None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:30.127447 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6ca379bf-e8d3-41db-85ad-f45fda91f7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:30.149176 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca379bf-e8d3-41db-85ad-f45fda91f7ae None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:30.168836 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e9993abe-b631-4f62-ad45-9e34d83dd414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'addc7988230a4022b4462fb9f31afcef', 'request_id': 'req-e9993abe-b631-4f62-ad45-9e34d83dd414'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:30.169998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:30.170114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:30.177259 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e9993abe-b631-4f62-ad45-9e34d83dd414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'addc7988230a4022b4462fb9f31afcef', 'request_id': 'req-e9993abe-b631-4f62-ad45-9e34d83dd414'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:30.186342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e9993abe-b631-4f62-ad45-9e34d83dd414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'addc7988230a4022b4462fb9f31afcef', 'request_id': 'req-e9993abe-b631-4f62-ad45-9e34d83dd414'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:30.198126 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e9993abe-b631-4f62-ad45-9e34d83dd414 admin admin] The token cache is being invalidate because project addc7988230a4022b4462fb9f31afcef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:30.198418 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e9993abe-b631-4f62-ad45-9e34d83dd414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:30.199084 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e9993abe-b631-4f62-ad45-9e34d83dd414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:30.200650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 995/2017] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:29 2022] DELETE /identity/v3/projects/addc7988230a4022b4462fb9f31afcef => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:30.203988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9993abe-b631-4f62-ad45-9e34d83dd414 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:30.205006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e9993abe-b631-4f62-ad45-9e34d83dd414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:30.217656 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b89db-cbbf-4b65-89d8-728fbe8205fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:30.218314 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b89db-cbbf-4b65-89d8-728fbe8205fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:30.218596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b89db-cbbf-4b65-89d8-728fbe8205fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:30.218873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b89db-cbbf-4b65-89d8-728fbe8205fb admin admin] PATH_INFO: `/v3/users/870b5b5b84464e2f9075502bf4a8f839` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:30.220054 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b89db-cbbf-4b65-89d8-728fbe8205fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=870b5b5b84464e2f9075502bf4a8f839)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:30.222032 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b89db-cbbf-4b65-89d8-728fbe8205fb admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:30.270094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-3e3b89db-cbbf-4b65-89d8-728fbe8205fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '870b5b5b84464e2f9075502bf4a8f839'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:30.270759 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-3e3b89db-cbbf-4b65-89d8-728fbe8205fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '870b5b5b84464e2f9075502bf4a8f839'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:30.271445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-3e3b89db-cbbf-4b65-89d8-728fbe8205fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '870b5b5b84464e2f9075502bf4a8f839'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:30.274581 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9993abe-b631-4f62-ad45-9e34d83dd414 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:30.279332 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-3e3b89db-cbbf-4b65-89d8-728fbe8205fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '870b5b5b84464e2f9075502bf4a8f839'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:30.280787 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66227dd-2a7e-45b5-b6a5-f4499c88a020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:30.281407 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66227dd-2a7e-45b5-b6a5-f4499c88a020 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:30.281688 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66227dd-2a7e-45b5-b6a5-f4499c88a020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:30.281960 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66227dd-2a7e-45b5-b6a5-f4499c88a020 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:30.282735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66227dd-2a7e-45b5-b6a5-f4499c88a020 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:30.284744 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66227dd-2a7e-45b5-b6a5-f4499c88a020 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:30.294913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-3e3b89db-cbbf-4b65-89d8-728fbe8205fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '870b5b5b84464e2f9075502bf4a8f839'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:30.310238 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1023/2018] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:30 2022] DELETE /identity/v3/users/870b5b5b84464e2f9075502bf4a8f839 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:30.314351 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b89db-cbbf-4b65-89d8-728fbe8205fb admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:30.318150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 996/2019] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:30 2022] POST /identity/v3/projects => generated 389 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:30.318287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3e3b89db-cbbf-4b65-89d8-728fbe8205fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:30.318379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef366957-4768-439c-99ca-9ee9cf8ef93b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:30.318379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef366957-4768-439c-99ca-9ee9cf8ef93b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:30.318379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef366957-4768-439c-99ca-9ee9cf8ef93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:30.338085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-ef366957-4768-439c-99ca-9ee9cf8ef93b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:30.339381 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ef366957-4768-439c-99ca-9ee9cf8ef93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:30.385270 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b89db-cbbf-4b65-89d8-728fbe8205fb admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:30.400735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 997/2020] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:30.404634 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef366957-4768-439c-99ca-9ee9cf8ef93b None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:30.406124 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ef366957-4768-439c-99ca-9ee9cf8ef93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:30.419025 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef366957-4768-439c-99ca-9ee9cf8ef93b None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:30.424867 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ffab2-b45d-495d-b3d6-4ba53c8e1b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:30.425494 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ffab2-b45d-495d-b3d6-4ba53c8e1b21 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:30.425776 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ffab2-b45d-495d-b3d6-4ba53c8e1b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:30.426052 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ffab2-b45d-495d-b3d6-4ba53c8e1b21 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:30.426710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ffab2-b45d-495d-b3d6-4ba53c8e1b21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:30.429492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ffab2-b45d-495d-b3d6-4ba53c8e1b21 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:30.430474 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:30.448979 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382854e-0622-49af-a909-89a2f908f2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:30.449630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382854e-0622-49af-a909-89a2f908f2c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:30.449911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382854e-0622-49af-a909-89a2f908f2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:30.450184 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382854e-0622-49af-a909-89a2f908f2c8 admin admin] PATH_INFO: `/v3/users/051ca8afa40f4c06a622328fd77402e8` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:30.451486 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382854e-0622-49af-a909-89a2f908f2c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=051ca8afa40f4c06a622328fd77402e8)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:30.453449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382854e-0622-49af-a909-89a2f908f2c8 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:30.458882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 998/2021] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:30 2022] POST /identity/v3/users => generated 301 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:30.458882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ffab2-b45d-495d-b3d6-4ba53c8e1b21 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:30.458882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3c0ffab2-b45d-495d-b3d6-4ba53c8e1b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:30.508287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-4382854e-0622-49af-a909-89a2f908f2c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '051ca8afa40f4c06a622328fd77402e8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:30.508950 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-4382854e-0622-49af-a909-89a2f908f2c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '051ca8afa40f4c06a622328fd77402e8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:30.510045 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-4382854e-0622-49af-a909-89a2f908f2c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '051ca8afa40f4c06a622328fd77402e8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:30.512581 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:30.518190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ffab2-b45d-495d-b3d6-4ba53c8e1b21 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:30.518459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-4382854e-0622-49af-a909-89a2f908f2c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '051ca8afa40f4c06a622328fd77402e8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:30.522516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1be4b-3c59-430d-bef0-c3b7c7d71069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:30.523542 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1be4b-3c59-430d-bef0-c3b7c7d71069 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:30.524005 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1be4b-3c59-430d-bef0-c3b7c7d71069 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:30.524278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1be4b-3c59-430d-bef0-c3b7c7d71069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:30.526684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d2f1be4b-3c59-430d-bef0-c3b7c7d71069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:30.531299 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-4382854e-0622-49af-a909-89a2f908f2c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '051ca8afa40f4c06a622328fd77402e8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:30.539491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1be4b-3c59-430d-bef0-c3b7c7d71069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:30.541219 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1be4b-3c59-430d-bef0-c3b7c7d71069 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:30.545556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1024/2022] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:30 2022] DELETE /identity/v3/users/051ca8afa40f4c06a622328fd77402e8 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:30.546442 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382854e-0622-49af-a909-89a2f908f2c8 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:30.549691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4382854e-0622-49af-a909-89a2f908f2c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:30.551621 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 999/2023] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:30.627575 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382854e-0622-49af-a909-89a2f908f2c8 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:30.632207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc71a50-d5d4-4b39-a2f5-6bd6f0a19a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:30.632830 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc71a50-d5d4-4b39-a2f5-6bd6f0a19a15 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:30.633109 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc71a50-d5d4-4b39-a2f5-6bd6f0a19a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:30.633375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc71a50-d5d4-4b39-a2f5-6bd6f0a19a15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:30.634028 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc71a50-d5d4-4b39-a2f5-6bd6f0a19a15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:30.635704 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc71a50-d5d4-4b39-a2f5-6bd6f0a19a15 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:30.647441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1025/2024] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:30.651822 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1be4b-3c59-430d-bef0-c3b7c7d71069 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:30.664066 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1be4b-3c59-430d-bef0-c3b7c7d71069 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:30.669396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc23358-fcb2-44e4-a3c5-24ebb7a079d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:30.670064 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc23358-fcb2-44e4-a3c5-24ebb7a079d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:30.670349 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc23358-fcb2-44e4-a3c5-24ebb7a079d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:30.670615 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc23358-fcb2-44e4-a3c5-24ebb7a079d8 admin admin] PATH_INFO: `/v3/projects/ab7e183cda534332aab566bf0ac08f62/users/4453284b51064a88a29f3ffec27d61ce/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:30.691404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc23358-fcb2-44e4-a3c5-24ebb7a079d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7e183cda534332aab566bf0ac08f62, user_id=4453284b51064a88a29f3ffec27d61ce, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:30.694159 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc23358-fcb2-44e4-a3c5-24ebb7a079d8 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:30.708808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1000/2025] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:30 2022] PUT /identity/v3/projects/ab7e183cda534332aab566bf0ac08f62/users/4453284b51064a88a29f3ffec27d61ce/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:30.712843 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc71a50-d5d4-4b39-a2f5-6bd6f0a19a15 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:30.723000 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc71a50-d5d4-4b39-a2f5-6bd6f0a19a15 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:30.784080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937298b-63a3-4e85-8e6a-ffd5c7f13d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:30.784699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937298b-63a3-4e85-8e6a-ffd5c7f13d57 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:30.784977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937298b-63a3-4e85-8e6a-ffd5c7f13d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:30.785220 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937298b-63a3-4e85-8e6a-ffd5c7f13d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:30.785918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937298b-63a3-4e85-8e6a-ffd5c7f13d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:30.787541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937298b-63a3-4e85-8e6a-ffd5c7f13d57 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:30.789038 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:30.800548 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1026/2026] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:30.805420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc23358-fcb2-44e4-a3c5-24ebb7a079d8 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:30.816254 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc23358-fcb2-44e4-a3c5-24ebb7a079d8 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:30.821186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69622cb0-974e-40f6-a75c-93e5d41b3dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:30.822017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622cb0-974e-40f6-a75c-93e5d41b3dff admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:30.822297 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622cb0-974e-40f6-a75c-93e5d41b3dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:30.822894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622cb0-974e-40f6-a75c-93e5d41b3dff admin admin] PATH_INFO: `/v3/projects/ab7e183cda534332aab566bf0ac08f62/users/4453284b51064a88a29f3ffec27d61ce/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:30.824843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69622cb0-974e-40f6-a75c-93e5d41b3dff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7e183cda534332aab566bf0ac08f62, user_id=4453284b51064a88a29f3ffec27d61ce, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:30.827579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69622cb0-974e-40f6-a75c-93e5d41b3dff admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:30.839902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1001/2027] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:30 2022] PUT /identity/v3/projects/ab7e183cda534332aab566bf0ac08f62/users/4453284b51064a88a29f3ffec27d61ce/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:30.851326 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7d673-d036-4584-a32e-ae7a3e3814d6 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:30.851651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7d673-d036-4584-a32e-ae7a3e3814d6 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:30.851978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7d673-d036-4584-a32e-ae7a3e3814d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:30.881114 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-5ed7d673-d036-4584-a32e-ae7a3e3814d6 None None] MFA Rules not processed for user `4453284b51064a88a29f3ffec27d61ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:30.882237 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-5ed7d673-d036-4584-a32e-ae7a3e3814d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:30.960221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1027/2028] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:30 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:30.983603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69622cb0-974e-40f6-a75c-93e5d41b3dff admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:30.983603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-69622cb0-974e-40f6-a75c-93e5d41b3dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:31.046449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7d673-d036-4584-a32e-ae7a3e3814d6 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:31.048328 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-5ed7d673-d036-4584-a32e-ae7a3e3814d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:31.061708 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69622cb0-974e-40f6-a75c-93e5d41b3dff admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:31.070327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e5124-1ce5-4439-b3bd-701dde25fda8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:31.072038 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e5124-1ce5-4439-b3bd-701dde25fda8 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:31.072777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e5124-1ce5-4439-b3bd-701dde25fda8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:31.073547 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e5124-1ce5-4439-b3bd-701dde25fda8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:31.088180 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e5124-1ce5-4439-b3bd-701dde25fda8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:31.090355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e5124-1ce5-4439-b3bd-701dde25fda8 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:31.104716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1002/2029] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:30 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:31.105821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e5124-1ce5-4439-b3bd-701dde25fda8 service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:31.107981 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3f3e5124-1ce5-4439-b3bd-701dde25fda8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:31.172631 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7d673-d036-4584-a32e-ae7a3e3814d6 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:31.186937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943eae4-44d8-4d67-91ef-651f20a4cfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:31.187782 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943eae4-44d8-4d67-91ef-651f20a4cfe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:31.188649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943eae4-44d8-4d67-91ef-651f20a4cfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:31.188649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943eae4-44d8-4d67-91ef-651f20a4cfe0 admin admin] PATH_INFO: `/v3/projects/310d40578da34892a86bf9daf0051ff2` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:31.190500 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943eae4-44d8-4d67-91ef-651f20a4cfe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=310d40578da34892a86bf9daf0051ff2)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:31.192572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943eae4-44d8-4d67-91ef-651f20a4cfe0 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:31.203057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e5124-1ce5-4439-b3bd-701dde25fda8 service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:31.210754 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39150a0e-e167-4254-be2c-3d1c7947bd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:31.211463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39150a0e-e167-4254-be2c-3d1c7947bd80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:31.211800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39150a0e-e167-4254-be2c-3d1c7947bd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:31.212303 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39150a0e-e167-4254-be2c-3d1c7947bd80 admin admin] PATH_INFO: `/v3/projects/4e8df44acf1249ef9f9ddeb8ef945f0c` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:31.213750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39150a0e-e167-4254-be2c-3d1c7947bd80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e8df44acf1249ef9f9ddeb8ef945f0c)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:31.216783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39150a0e-e167-4254-be2c-3d1c7947bd80 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:31.271922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-8943eae4-44d8-4d67-91ef-651f20a4cfe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '310d40578da34892a86bf9daf0051ff2', 'request_id': 'req-8943eae4-44d8-4d67-91ef-651f20a4cfe0'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.285874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-8943eae4-44d8-4d67-91ef-651f20a4cfe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '310d40578da34892a86bf9daf0051ff2', 'request_id': 'req-8943eae4-44d8-4d67-91ef-651f20a4cfe0'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.295928 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-8943eae4-44d8-4d67-91ef-651f20a4cfe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '310d40578da34892a86bf9daf0051ff2', 'request_id': 'req-8943eae4-44d8-4d67-91ef-651f20a4cfe0'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.302863 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-39150a0e-e167-4254-be2c-3d1c7947bd80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e8df44acf1249ef9f9ddeb8ef945f0c', 'request_id': 'req-39150a0e-e167-4254-be2c-3d1c7947bd80'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.309641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:31.312037 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-39150a0e-e167-4254-be2c-3d1c7947bd80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e8df44acf1249ef9f9ddeb8ef945f0c', 'request_id': 'req-39150a0e-e167-4254-be2c-3d1c7947bd80'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.317450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-8943eae4-44d8-4d67-91ef-651f20a4cfe0 admin admin] The token cache is being invalidate because project 310d40578da34892a86bf9daf0051ff2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:31.317809 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-8943eae4-44d8-4d67-91ef-651f20a4cfe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.318822 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-8943eae4-44d8-4d67-91ef-651f20a4cfe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.323689 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1028/2030] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:31 2022] DELETE /identity/v3/projects/310d40578da34892a86bf9daf0051ff2 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:31.324623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-39150a0e-e167-4254-be2c-3d1c7947bd80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e8df44acf1249ef9f9ddeb8ef945f0c', 'request_id': 'req-39150a0e-e167-4254-be2c-3d1c7947bd80'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.328359 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943eae4-44d8-4d67-91ef-651f20a4cfe0 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:31.333348 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8943eae4-44d8-4d67-91ef-651f20a4cfe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:31.341136 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-39150a0e-e167-4254-be2c-3d1c7947bd80 admin admin] The token cache is being invalidate because project 4e8df44acf1249ef9f9ddeb8ef945f0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:31.341430 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-39150a0e-e167-4254-be2c-3d1c7947bd80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.342093 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-39150a0e-e167-4254-be2c-3d1c7947bd80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.343976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1003/2031] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:31 2022] DELETE /identity/v3/projects/4e8df44acf1249ef9f9ddeb8ef945f0c => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:31.349094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d828b7f-844c-45cd-8bb1-cd0c023311c2 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:31.349384 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d828b7f-844c-45cd-8bb1-cd0c023311c2 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:31.349664 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d828b7f-844c-45cd-8bb1-cd0c023311c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:31.375635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:31.377330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-6d828b7f-844c-45cd-8bb1-cd0c023311c2 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:31.378906 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-6d828b7f-844c-45cd-8bb1-cd0c023311c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:31.446988 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943eae4-44d8-4d67-91ef-651f20a4cfe0 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:31.449409 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1004/2032] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:31.450264 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d828b7f-844c-45cd-8bb1-cd0c023311c2 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:31.451546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-6d828b7f-844c-45cd-8bb1-cd0c023311c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:31.451882 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad75d00-f9a0-44af-ac3c-2b98997a6faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:31.452522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad75d00-f9a0-44af-ac3c-2b98997a6faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:31.452800 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad75d00-f9a0-44af-ac3c-2b98997a6faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:31.453070 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad75d00-f9a0-44af-ac3c-2b98997a6faa admin admin] PATH_INFO: `/v3/users/a006a62bed3a459a9a280b8fabde4f8e` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:31.454229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad75d00-f9a0-44af-ac3c-2b98997a6faa admin admin] RBAC: Authorizing `identity:delete_user(user_id=a006a62bed3a459a9a280b8fabde4f8e)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:31.456264 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad75d00-f9a0-44af-ac3c-2b98997a6faa admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:31.466268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d828b7f-844c-45cd-8bb1-cd0c023311c2 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:31.472492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cd43f-031e-4bdf-badf-cb0a594d39c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:31.473271 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cd43f-031e-4bdf-badf-cb0a594d39c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:31.473671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cd43f-031e-4bdf-badf-cb0a594d39c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:31.474013 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cd43f-031e-4bdf-badf-cb0a594d39c2 admin admin] PATH_INFO: `/v3/users/4ce088523a9e4e64994392f1d41e2545` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:31.475235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909cd43f-031e-4bdf-badf-cb0a594d39c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ce088523a9e4e64994392f1d41e2545)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:31.477542 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909cd43f-031e-4bdf-badf-cb0a594d39c2 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:31.524389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fad75d00-f9a0-44af-ac3c-2b98997a6faa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a006a62bed3a459a9a280b8fabde4f8e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.525304 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fad75d00-f9a0-44af-ac3c-2b98997a6faa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a006a62bed3a459a9a280b8fabde4f8e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.526138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fad75d00-f9a0-44af-ac3c-2b98997a6faa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a006a62bed3a459a9a280b8fabde4f8e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.533750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fad75d00-f9a0-44af-ac3c-2b98997a6faa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a006a62bed3a459a9a280b8fabde4f8e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.542075 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-909cd43f-031e-4bdf-badf-cb0a594d39c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ce088523a9e4e64994392f1d41e2545'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.542748 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-909cd43f-031e-4bdf-badf-cb0a594d39c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ce088523a9e4e64994392f1d41e2545'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.543448 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-909cd43f-031e-4bdf-badf-cb0a594d39c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ce088523a9e4e64994392f1d41e2545'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.547983 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fad75d00-f9a0-44af-ac3c-2b98997a6faa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a006a62bed3a459a9a280b8fabde4f8e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.550727 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-909cd43f-031e-4bdf-badf-cb0a594d39c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ce088523a9e4e64994392f1d41e2545'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.562222 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1029/2033] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:31 2022] DELETE /identity/v3/users/a006a62bed3a459a9a280b8fabde4f8e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:31.563509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d4a49-8da3-49ac-ae62-7b9480a88654 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:31.563798 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d4a49-8da3-49ac-ae62-7b9480a88654 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:31.564072 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d4a49-8da3-49ac-ae62-7b9480a88654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:31.564847 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-909cd43f-031e-4bdf-badf-cb0a594d39c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ce088523a9e4e64994392f1d41e2545'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:31.587119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1005/2034] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:31 2022] DELETE /identity/v3/users/4ce088523a9e4e64994392f1d41e2545 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:31.588294 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-f63d4a49-8da3-49ac-ae62-7b9480a88654 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:31.588728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfbc93-0aae-4084-9620-4f2971a5bb7a None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:31.589125 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfbc93-0aae-4084-9620-4f2971a5bb7a None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:31.589367 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-f63d4a49-8da3-49ac-ae62-7b9480a88654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:31.589625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfbc93-0aae-4084-9620-4f2971a5bb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:31.609625 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-14cfbc93-0aae-4084-9620-4f2971a5bb7a None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:31.610985 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-14cfbc93-0aae-4084-9620-4f2971a5bb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:31.672605 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1030/2035] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:31.676899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:31.677760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d4a49-8da3-49ac-ae62-7b9480a88654 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:31.692675 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d4a49-8da3-49ac-ae62-7b9480a88654 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:31.695736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1006/2036] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:31.699294 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9a188-bf6a-4618-aafe-0c09b08e2164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:31.699960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9a188-bf6a-4618-aafe-0c09b08e2164 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:31.700234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9a188-bf6a-4618-aafe-0c09b08e2164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:31.700504 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9a188-bf6a-4618-aafe-0c09b08e2164 admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:31.701170 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d9a188-bf6a-4618-aafe-0c09b08e2164 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:31.701468 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d9a188-bf6a-4618-aafe-0c09b08e2164 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:31.702414 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfbc93-0aae-4084-9620-4f2971a5bb7a None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:31.703636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d9a188-bf6a-4618-aafe-0c09b08e2164 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:31.703907 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-14cfbc93-0aae-4084-9620-4f2971a5bb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:31.718968 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1031/2037] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:31.726000 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9a188-bf6a-4618-aafe-0c09b08e2164 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:31.736539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9a188-bf6a-4618-aafe-0c09b08e2164 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:31.741702 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4151f49-e41c-4bc1-b374-fd53ae137f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:31.742206 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4151f49-e41c-4bc1-b374-fd53ae137f8c admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:31.742478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4151f49-e41c-4bc1-b374-fd53ae137f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:31.742801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4151f49-e41c-4bc1-b374-fd53ae137f8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:31.743368 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4151f49-e41c-4bc1-b374-fd53ae137f8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:31.746206 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4151f49-e41c-4bc1-b374-fd53ae137f8c admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:31.764474 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:31.773224 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfbc93-0aae-4084-9620-4f2971a5bb7a None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:31.777637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d74531-8ea8-4f46-ac01-deb771471ac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:31.778369 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d74531-8ea8-4f46-ac01-deb771471ac4 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:31.778665 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d74531-8ea8-4f46-ac01-deb771471ac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:31.778910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d74531-8ea8-4f46-ac01-deb771471ac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:31.782799 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1032/2038] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:31 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:31.786146 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4151f49-e41c-4bc1-b374-fd53ae137f8c admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:31.792711 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d74531-8ea8-4f46-ac01-deb771471ac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:31.794307 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d74531-8ea8-4f46-ac01-deb771471ac4 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:31.804713 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1007/2039] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:31.852496 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4151f49-e41c-4bc1-b374-fd53ae137f8c admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:31.857300 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b8d95-4e67-4167-8c3d-2977658d1d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:31.857914 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b8d95-4e67-4167-8c3d-2977658d1d44 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:31.858191 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b8d95-4e67-4167-8c3d-2977658d1d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:31.858459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b8d95-4e67-4167-8c3d-2977658d1d44 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:31.859085 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b8d95-4e67-4167-8c3d-2977658d1d44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:31.861095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b8d95-4e67-4167-8c3d-2977658d1d44 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:31.884826 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1033/2040] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:31 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:31.891182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d74531-8ea8-4f46-ac01-deb771471ac4 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:31.903245 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d74531-8ea8-4f46-ac01-deb771471ac4 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:31.908292 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f1842-f57b-4064-accb-759362427c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:31.909004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f1842-f57b-4064-accb-759362427c3e admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:31.909221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f1842-f57b-4064-accb-759362427c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:31.909546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f1842-f57b-4064-accb-759362427c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:31.910176 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f1842-f57b-4064-accb-759362427c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:31.911902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f1842-f57b-4064-accb-759362427c3e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:31.919709 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b8d95-4e67-4167-8c3d-2977658d1d44 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:31.921673 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9a8b8d95-4e67-4167-8c3d-2977658d1d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:31.922843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1008/2041] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:31.927195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f1842-f57b-4064-accb-759362427c3e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:31.937334 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b8d95-4e67-4167-8c3d-2977658d1d44 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:31.938804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f1842-f57b-4064-accb-759362427c3e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:31.948065 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817566e-cf91-4a3c-89e8-455b40ed97d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:31.948716 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817566e-cf91-4a3c-89e8-455b40ed97d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:31.949080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817566e-cf91-4a3c-89e8-455b40ed97d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:31.949284 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817566e-cf91-4a3c-89e8-455b40ed97d8 admin admin] PATH_INFO: `/v3/projects/e7bf44a6062d427f8ac5c97228702755` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:31.950347 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3407ba-1a12-4299-b864-c49c427ed6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:31.950780 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3407ba-1a12-4299-b864-c49c427ed6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:31.951050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3407ba-1a12-4299-b864-c49c427ed6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:31.951312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3407ba-1a12-4299-b864-c49c427ed6a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:31.952027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3407ba-1a12-4299-b864-c49c427ed6a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:31.954396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817566e-cf91-4a3c-89e8-455b40ed97d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7bf44a6062d427f8ac5c97228702755)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:31.957662 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817566e-cf91-4a3c-89e8-455b40ed97d8 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:31.959538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3407ba-1a12-4299-b864-c49c427ed6a2 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:31.972337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1009/2042] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:31.976532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3407ba-1a12-4299-b864-c49c427ed6a2 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:31.990300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3407ba-1a12-4299-b864-c49c427ed6a2 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:31.995204 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aaafed-2f56-422d-9215-945a91ddaf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:31.995906 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aaafed-2f56-422d-9215-945a91ddaf1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:31.996337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aaafed-2f56-422d-9215-945a91ddaf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:31.996737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aaafed-2f56-422d-9215-945a91ddaf1a admin admin] PATH_INFO: `/v3/projects/4a73dcb1b32b4d8597b76cb0b27aefcf/users/c6a6dae1083b4729b237dcf83c75a550/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.020614 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aaafed-2f56-422d-9215-945a91ddaf1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a73dcb1b32b4d8597b76cb0b27aefcf, user_id=c6a6dae1083b4729b237dcf83c75a550, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:32.026420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aaafed-2f56-422d-9215-945a91ddaf1a admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:32.048243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-9817566e-cf91-4a3c-89e8-455b40ed97d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7bf44a6062d427f8ac5c97228702755', 'request_id': 'req-9817566e-cf91-4a3c-89e8-455b40ed97d8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:32.053326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1010/2043] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:31 2022] PUT /identity/v3/projects/4a73dcb1b32b4d8597b76cb0b27aefcf/users/c6a6dae1083b4729b237dcf83c75a550/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:32.056312 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-9817566e-cf91-4a3c-89e8-455b40ed97d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7bf44a6062d427f8ac5c97228702755', 'request_id': 'req-9817566e-cf91-4a3c-89e8-455b40ed97d8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:32.064485 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac4a26-46c0-4fc2-b45b-11689fa21125 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.064773 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac4a26-46c0-4fc2-b45b-11689fa21125 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.065051 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac4a26-46c0-4fc2-b45b-11689fa21125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.066000 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-9817566e-cf91-4a3c-89e8-455b40ed97d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7bf44a6062d427f8ac5c97228702755', 'request_id': 'req-9817566e-cf91-4a3c-89e8-455b40ed97d8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:32.080415 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-9817566e-cf91-4a3c-89e8-455b40ed97d8 admin admin] The token cache is being invalidate because project e7bf44a6062d427f8ac5c97228702755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:32.081179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-9817566e-cf91-4a3c-89e8-455b40ed97d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:32.082215 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-9817566e-cf91-4a3c-89e8-455b40ed97d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:32.084190 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1034/2044] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:31 2022] DELETE /identity/v3/projects/e7bf44a6062d427f8ac5c97228702755 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:32.088711 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585501d-f2be-4902-80ed-8dc2260182f8 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.089217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585501d-f2be-4902-80ed-8dc2260182f8 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.089486 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585501d-f2be-4902-80ed-8dc2260182f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.095921 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-e6ac4a26-46c0-4fc2-b45b-11689fa21125 None None] MFA Rules not processed for user `c6a6dae1083b4729b237dcf83c75a550`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:32.097263 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e6ac4a26-46c0-4fc2-b45b-11689fa21125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:32.110795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-4585501d-f2be-4902-80ed-8dc2260182f8 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:32.112154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4585501d-f2be-4902-80ed-8dc2260182f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:32.171211 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1011/2045] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:32.175428 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac4a26-46c0-4fc2-b45b-11689fa21125 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:32.177013 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e6ac4a26-46c0-4fc2-b45b-11689fa21125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:32.188244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1035/2046] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:32.189424 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585501d-f2be-4902-80ed-8dc2260182f8 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:32.190946 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4585501d-f2be-4902-80ed-8dc2260182f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:32.192382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac4a26-46c0-4fc2-b45b-11689fa21125 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:32.197862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da2388-692a-4fef-9ffa-53bdec64735c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:32.198566 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da2388-692a-4fef-9ffa-53bdec64735c admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.198915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da2388-692a-4fef-9ffa-53bdec64735c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.199369 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da2388-692a-4fef-9ffa-53bdec64735c admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.200090 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da2388-692a-4fef-9ffa-53bdec64735c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:32.202274 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da2388-692a-4fef-9ffa-53bdec64735c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:32.208340 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585501d-f2be-4902-80ed-8dc2260182f8 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:32.216880 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba9839-d487-4db2-8d01-fdab036ddadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:32.217544 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba9839-d487-4db2-8d01-fdab036ddadd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.217819 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba9839-d487-4db2-8d01-fdab036ddadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.218200 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba9839-d487-4db2-8d01-fdab036ddadd admin admin] PATH_INFO: `/v3/projects/767bd9c6402e4ef7bd283c56eb8b6c98` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.219298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba9839-d487-4db2-8d01-fdab036ddadd admin admin] RBAC: Authorizing `identity:delete_project(project_id=767bd9c6402e4ef7bd283c56eb8b6c98)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:32.221262 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba9839-d487-4db2-8d01-fdab036ddadd admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:32.225175 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:32.247973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1012/2047] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:32 2022] POST /identity/v3/projects => generated 365 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:32.247973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01f38b-13da-49bf-bcf0-702f5fe851c2 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.247973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01f38b-13da-49bf-bcf0-702f5fe851c2 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.247973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01f38b-13da-49bf-bcf0-702f5fe851c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.268374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-3c01f38b-13da-49bf-bcf0-702f5fe851c2 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:32.269661 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3c01f38b-13da-49bf-bcf0-702f5fe851c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:32.283060 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-09ba9839-d487-4db2-8d01-fdab036ddadd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '767bd9c6402e4ef7bd283c56eb8b6c98', 'request_id': 'req-09ba9839-d487-4db2-8d01-fdab036ddadd'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:32.288493 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-09ba9839-d487-4db2-8d01-fdab036ddadd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '767bd9c6402e4ef7bd283c56eb8b6c98', 'request_id': 'req-09ba9839-d487-4db2-8d01-fdab036ddadd'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:32.288862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:32.299343 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-09ba9839-d487-4db2-8d01-fdab036ddadd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '767bd9c6402e4ef7bd283c56eb8b6c98', 'request_id': 'req-09ba9839-d487-4db2-8d01-fdab036ddadd'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:32.314935 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-09ba9839-d487-4db2-8d01-fdab036ddadd admin admin] The token cache is being invalidate because project 767bd9c6402e4ef7bd283c56eb8b6c98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:32.315255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-09ba9839-d487-4db2-8d01-fdab036ddadd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:32.316036 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-09ba9839-d487-4db2-8d01-fdab036ddadd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:32.317869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1036/2048] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:32 2022] DELETE /identity/v3/projects/767bd9c6402e4ef7bd283c56eb8b6c98 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:32.318688 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba9839-d487-4db2-8d01-fdab036ddadd admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:32.320337 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-09ba9839-d487-4db2-8d01-fdab036ddadd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:32.347024 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1013/2049] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:32.347481 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01f38b-13da-49bf-bcf0-702f5fe851c2 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:32.348872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3c01f38b-13da-49bf-bcf0-702f5fe851c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:32.363227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01f38b-13da-49bf-bcf0-702f5fe851c2 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:32.368139 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90083b51-4be7-44f1-b06c-4eb03f7406ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:32.368832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90083b51-4be7-44f1-b06c-4eb03f7406ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.369113 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90083b51-4be7-44f1-b06c-4eb03f7406ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.369393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90083b51-4be7-44f1-b06c-4eb03f7406ba admin admin] PATH_INFO: `/v3/users/4453284b51064a88a29f3ffec27d61ce` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.370562 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90083b51-4be7-44f1-b06c-4eb03f7406ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=4453284b51064a88a29f3ffec27d61ce)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:32.372565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90083b51-4be7-44f1-b06c-4eb03f7406ba admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:32.394410 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba9839-d487-4db2-8d01-fdab036ddadd admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:32.399113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa86e7-8a2a-49d5-ab9f-799bc7bec5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:32.399827 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa86e7-8a2a-49d5-ab9f-799bc7bec5a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.400125 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa86e7-8a2a-49d5-ab9f-799bc7bec5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.400424 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa86e7-8a2a-49d5-ab9f-799bc7bec5a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.401056 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa86e7-8a2a-49d5-ab9f-799bc7bec5a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:32.403018 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa86e7-8a2a-49d5-ab9f-799bc7bec5a9 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:32.426439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-90083b51-4be7-44f1-b06c-4eb03f7406ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4453284b51064a88a29f3ffec27d61ce'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:32.427118 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-90083b51-4be7-44f1-b06c-4eb03f7406ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4453284b51064a88a29f3ffec27d61ce'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:32.428190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-90083b51-4be7-44f1-b06c-4eb03f7406ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4453284b51064a88a29f3ffec27d61ce'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:32.434402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1037/2050] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:32 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:32.434402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa86e7-8a2a-49d5-ab9f-799bc7bec5a9 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:32.434715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9aaa86e7-8a2a-49d5-ab9f-799bc7bec5a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:32.436247 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-90083b51-4be7-44f1-b06c-4eb03f7406ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4453284b51064a88a29f3ffec27d61ce'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:32.449448 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa86e7-8a2a-49d5-ab9f-799bc7bec5a9 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:32.451167 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-90083b51-4be7-44f1-b06c-4eb03f7406ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4453284b51064a88a29f3ffec27d61ce'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:32.454670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c13cf3-2bd9-4fca-b596-37c31d5b9cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:32.455358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c13cf3-2bd9-4fca-b596-37c31d5b9cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.455734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c13cf3-2bd9-4fca-b596-37c31d5b9cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.456079 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c13cf3-2bd9-4fca-b596-37c31d5b9cc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.457110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c13cf3-2bd9-4fca-b596-37c31d5b9cc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:32.457407 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c13cf3-2bd9-4fca-b596-37c31d5b9cc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:32.459184 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c13cf3-2bd9-4fca-b596-37c31d5b9cc0 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:32.471384 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1014/2051] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:32 2022] DELETE /identity/v3/users/4453284b51064a88a29f3ffec27d61ce => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:32.472560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90083b51-4be7-44f1-b06c-4eb03f7406ba admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:32.475488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1038/2052] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:32.475622 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-90083b51-4be7-44f1-b06c-4eb03f7406ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:32.477872 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c13cf3-2bd9-4fca-b596-37c31d5b9cc0 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:32.488673 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c13cf3-2bd9-4fca-b596-37c31d5b9cc0 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:32.550928 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90083b51-4be7-44f1-b06c-4eb03f7406ba admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:32.558786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e1e12-5f1e-4452-9770-8ee1d49c2308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:32.559724 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e1e12-5f1e-4452-9770-8ee1d49c2308 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.560109 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e1e12-5f1e-4452-9770-8ee1d49c2308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.560927 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e1e12-5f1e-4452-9770-8ee1d49c2308 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.561696 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e1e12-5f1e-4452-9770-8ee1d49c2308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:32.563532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e1e12-5f1e-4452-9770-8ee1d49c2308 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:32.576099 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1015/2053] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:32.587761 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e1e12-5f1e-4452-9770-8ee1d49c2308 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:32.592116 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eca44b-17b2-4862-bb92-b87566c3875f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:32.592559 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eca44b-17b2-4862-bb92-b87566c3875f admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.592998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eca44b-17b2-4862-bb92-b87566c3875f admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.593368 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eca44b-17b2-4862-bb92-b87566c3875f admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.594231 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eca44b-17b2-4862-bb92-b87566c3875f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:32.596310 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eca44b-17b2-4862-bb92-b87566c3875f admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:32.600465 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e1e12-5f1e-4452-9770-8ee1d49c2308 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:32.605969 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f9b36-09c7-4832-9bd2-9501075f690e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:32.606639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f9b36-09c7-4832-9bd2-9501075f690e admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.606914 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f9b36-09c7-4832-9bd2-9501075f690e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.607181 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f9b36-09c7-4832-9bd2-9501075f690e admin admin] PATH_INFO: `/v3/projects/6972c61a598e4a798aa9c5832256108d/users/053a8ad2ad004dbeb09f66c6273c8f05/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.615293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:32.630095 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f9b36-09c7-4832-9bd2-9501075f690e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6972c61a598e4a798aa9c5832256108d, user_id=053a8ad2ad004dbeb09f66c6273c8f05, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:32.633109 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f9b36-09c7-4832-9bd2-9501075f690e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:32.637207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1039/2054] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:32 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:32.639752 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eca44b-17b2-4862-bb92-b87566c3875f admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:32.668073 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1016/2055] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:32 2022] PUT /identity/v3/projects/6972c61a598e4a798aa9c5832256108d/users/053a8ad2ad004dbeb09f66c6273c8f05/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:32.668073 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f9b36-09c7-4832-9bd2-9501075f690e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:32.677279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:32.729242 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eca44b-17b2-4862-bb92-b87566c3875f admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:32.757103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f9b36-09c7-4832-9bd2-9501075f690e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:32.763352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65226bb7-928d-478d-9af8-abc3ca40de3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:32.764188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65226bb7-928d-478d-9af8-abc3ca40de3e admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.764472 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65226bb7-928d-478d-9af8-abc3ca40de3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.764769 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65226bb7-928d-478d-9af8-abc3ca40de3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.765450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65226bb7-928d-478d-9af8-abc3ca40de3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:32.767085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65226bb7-928d-478d-9af8-abc3ca40de3e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:32.782189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1017/2056] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:32.783261 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65226bb7-928d-478d-9af8-abc3ca40de3e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:32.796226 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65226bb7-928d-478d-9af8-abc3ca40de3e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:32.801045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddc6ae-556c-4438-a2cc-82c8e1ec590b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:32.801188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87193a8-5afe-4f81-9562-1388ef6ba448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:32.801734 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddc6ae-556c-4438-a2cc-82c8e1ec590b admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.801861 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87193a8-5afe-4f81-9562-1388ef6ba448 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.802023 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddc6ae-556c-4438-a2cc-82c8e1ec590b admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.802224 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87193a8-5afe-4f81-9562-1388ef6ba448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.802325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddc6ae-556c-4438-a2cc-82c8e1ec590b admin admin] PATH_INFO: `/v3/projects/6972c61a598e4a798aa9c5832256108d/users/053a8ad2ad004dbeb09f66c6273c8f05/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.802596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87193a8-5afe-4f81-9562-1388ef6ba448 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.803443 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87193a8-5afe-4f81-9562-1388ef6ba448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:32.804434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ddc6ae-556c-4438-a2cc-82c8e1ec590b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6972c61a598e4a798aa9c5832256108d, user_id=053a8ad2ad004dbeb09f66c6273c8f05, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:32.805550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87193a8-5afe-4f81-9562-1388ef6ba448 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:32.807570 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ddc6ae-556c-4438-a2cc-82c8e1ec590b admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:32.820887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1018/2057] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:32 2022] PUT /identity/v3/projects/6972c61a598e4a798aa9c5832256108d/users/053a8ad2ad004dbeb09f66c6273c8f05/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:32.835298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3d49e-ea54-4c53-80c4-5a1b1f29aa7c None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.835777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3d49e-ea54-4c53-80c4-5a1b1f29aa7c None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.836057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3d49e-ea54-4c53-80c4-5a1b1f29aa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.846662 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1040/2058] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:32 2022] POST /identity/v3/users => generated 292 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:32.848008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87193a8-5afe-4f81-9562-1388ef6ba448 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:32.858703 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87193a8-5afe-4f81-9562-1388ef6ba448 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:32.878517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-8fc3d49e-ea54-4c53-80c4-5a1b1f29aa7c None None] MFA Rules not processed for user `053a8ad2ad004dbeb09f66c6273c8f05`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:32.879960 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8fc3d49e-ea54-4c53-80c4-5a1b1f29aa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:32.930109 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af7405-fb75-4a02-a355-adea786a3709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:32.930681 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af7405-fb75-4a02-a355-adea786a3709 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.931114 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af7405-fb75-4a02-a355-adea786a3709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.931555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af7405-fb75-4a02-a355-adea786a3709 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.932227 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af7405-fb75-4a02-a355-adea786a3709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:32.933842 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af7405-fb75-4a02-a355-adea786a3709 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:32.944186 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1041/2059] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:32.944999 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af7405-fb75-4a02-a355-adea786a3709 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:32.946678 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e9af7405-fb75-4a02-a355-adea786a3709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:32.959435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af7405-fb75-4a02-a355-adea786a3709 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:32.960954 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1019/2060] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:32.961912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3d49e-ea54-4c53-80c4-5a1b1f29aa7c None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:32.965727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004f5e9-2336-4be9-8ee8-0f4a94a7b045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:32.966373 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004f5e9-2336-4be9-8ee8-0f4a94a7b045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.966650 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004f5e9-2336-4be9-8ee8-0f4a94a7b045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.966922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004f5e9-2336-4be9-8ee8-0f4a94a7b045 admin admin] PATH_INFO: `/v3/projects/ab7e183cda534332aab566bf0ac08f62` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.968229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004f5e9-2336-4be9-8ee8-0f4a94a7b045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab7e183cda534332aab566bf0ac08f62)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:32.970791 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004f5e9-2336-4be9-8ee8-0f4a94a7b045 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:32.972910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3d49e-ea54-4c53-80c4-5a1b1f29aa7c None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:32.978412 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe3d6f-b967-46b3-bee2-1ad5a3af1777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:32.979029 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe3d6f-b967-46b3-bee2-1ad5a3af1777 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:32.979305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe3d6f-b967-46b3-bee2-1ad5a3af1777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:32.979607 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe3d6f-b967-46b3-bee2-1ad5a3af1777 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:32.980258 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe3d6f-b967-46b3-bee2-1ad5a3af1777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:32.981867 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe3d6f-b967-46b3-bee2-1ad5a3af1777 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:32.993350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1020/2061] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:32.994221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe3d6f-b967-46b3-bee2-1ad5a3af1777 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:32.996103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9fbe3d6f-b967-46b3-bee2-1ad5a3af1777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:32.996467 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:33.022712 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-2004f5e9-2336-4be9-8ee8-0f4a94a7b045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab7e183cda534332aab566bf0ac08f62', 'request_id': 'req-2004f5e9-2336-4be9-8ee8-0f4a94a7b045'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:33.029164 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-2004f5e9-2336-4be9-8ee8-0f4a94a7b045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab7e183cda534332aab566bf0ac08f62', 'request_id': 'req-2004f5e9-2336-4be9-8ee8-0f4a94a7b045'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:33.037161 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-2004f5e9-2336-4be9-8ee8-0f4a94a7b045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab7e183cda534332aab566bf0ac08f62', 'request_id': 'req-2004f5e9-2336-4be9-8ee8-0f4a94a7b045'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:33.049988 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-2004f5e9-2336-4be9-8ee8-0f4a94a7b045 admin admin] The token cache is being invalidate because project ab7e183cda534332aab566bf0ac08f62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:33.050286 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-2004f5e9-2336-4be9-8ee8-0f4a94a7b045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:33.050919 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-2004f5e9-2336-4be9-8ee8-0f4a94a7b045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:33.052495 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1042/2062] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:32 2022] DELETE /identity/v3/projects/ab7e183cda534332aab566bf0ac08f62 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:33.053311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004f5e9-2336-4be9-8ee8-0f4a94a7b045 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:33.054572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-2004f5e9-2336-4be9-8ee8-0f4a94a7b045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:33.062218 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe3d6f-b967-46b3-bee2-1ad5a3af1777 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:33.080387 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:33.122421 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47683fbb-1a1a-4cca-a61d-a5e21e73da34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:33.123262 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47683fbb-1a1a-4cca-a61d-a5e21e73da34 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.123633 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47683fbb-1a1a-4cca-a61d-a5e21e73da34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.123975 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47683fbb-1a1a-4cca-a61d-a5e21e73da34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.125850 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-47683fbb-1a1a-4cca-a61d-a5e21e73da34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:33.129666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004f5e9-2336-4be9-8ee8-0f4a94a7b045 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:33.134726 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88fd32-6310-4f47-a8e5-71f2e2823f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:33.135386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88fd32-6310-4f47-a8e5-71f2e2823f54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.135688 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88fd32-6310-4f47-a8e5-71f2e2823f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.135959 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88fd32-6310-4f47-a8e5-71f2e2823f54 admin admin] PATH_INFO: `/v3/projects/857e3448f8ec445bbba6589f2acde255/users/b740bcefdbd243fcbe1e49442275d347/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.150659 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88fd32-6310-4f47-a8e5-71f2e2823f54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=857e3448f8ec445bbba6589f2acde255, user_id=b740bcefdbd243fcbe1e49442275d347, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:33.153540 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88fd32-6310-4f47-a8e5-71f2e2823f54 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:33.167664 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1043/2063] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:33 2022] PUT /identity/v3/projects/857e3448f8ec445bbba6589f2acde255/users/b740bcefdbd243fcbe1e49442275d347/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:33.168911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057dcb1-5b1f-4e08-8b94-6ebe2f1906cb None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.169180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057dcb1-5b1f-4e08-8b94-6ebe2f1906cb None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.169447 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057dcb1-5b1f-4e08-8b94-6ebe2f1906cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.189741 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-3057dcb1-5b1f-4e08-8b94-6ebe2f1906cb None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:33.190851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3057dcb1-5b1f-4e08-8b94-6ebe2f1906cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:33.193311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47683fbb-1a1a-4cca-a61d-a5e21e73da34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:33.194771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47683fbb-1a1a-4cca-a61d-a5e21e73da34 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:33.205140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1021/2064] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:33.206422 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9232760-ee62-477d-84f0-12e0782dfb18 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.207014 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9232760-ee62-477d-84f0-12e0782dfb18 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.207299 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9232760-ee62-477d-84f0-12e0782dfb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.240518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-d9232760-ee62-477d-84f0-12e0782dfb18 None None] MFA Rules not processed for user `b740bcefdbd243fcbe1e49442275d347`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:33.241908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d9232760-ee62-477d-84f0-12e0782dfb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:33.257525 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1044/2065] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:33.265720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c8d28-aa07-4f12-bac1-1301cf67202e None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.265933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c8d28-aa07-4f12-bac1-1301cf67202e None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.266134 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c8d28-aa07-4f12-bac1-1301cf67202e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.296093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-853c8d28-aa07-4f12-bac1-1301cf67202e None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:33.297325 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-853c8d28-aa07-4f12-bac1-1301cf67202e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:33.310078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1045/2066] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:33.311159 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c8d28-aa07-4f12-bac1-1301cf67202e None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:33.314588 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-853c8d28-aa07-4f12-bac1-1301cf67202e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:33.336365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1022/2067] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:33.337085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9232760-ee62-477d-84f0-12e0782dfb18 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:33.347454 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9232760-ee62-477d-84f0-12e0782dfb18 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:33.352426 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12c75d-819c-485d-ae32-04d7993051f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:33.353201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12c75d-819c-485d-ae32-04d7993051f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.353613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12c75d-819c-485d-ae32-04d7993051f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.354054 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12c75d-819c-485d-ae32-04d7993051f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.354778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12c75d-819c-485d-ae32-04d7993051f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:33.355154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12c75d-819c-485d-ae32-04d7993051f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:33.356906 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12c75d-819c-485d-ae32-04d7993051f7 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:33.359245 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:33.372063 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1023/2068] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:33.373196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12c75d-819c-485d-ae32-04d7993051f7 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:33.383574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12c75d-819c-485d-ae32-04d7993051f7 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:33.388203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45223686-ae99-4321-8b41-8a9215b2493f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:33.388976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45223686-ae99-4321-8b41-8a9215b2493f admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.389258 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45223686-ae99-4321-8b41-8a9215b2493f admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.389527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45223686-ae99-4321-8b41-8a9215b2493f admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.390147 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45223686-ae99-4321-8b41-8a9215b2493f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:33.391181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c8d28-aa07-4f12-bac1-1301cf67202e None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:33.392105 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45223686-ae99-4321-8b41-8a9215b2493f admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:33.395092 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb982c39-b672-4821-9772-df2a003c793d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:33.395737 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb982c39-b672-4821-9772-df2a003c793d service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.396017 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb982c39-b672-4821-9772-df2a003c793d service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.396435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb982c39-b672-4821-9772-df2a003c793d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.398272 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-eb982c39-b672-4821-9772-df2a003c793d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:33.424285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1024/2069] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:33 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:33.424285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45223686-ae99-4321-8b41-8a9215b2493f admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:33.451182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:33.469275 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb982c39-b672-4821-9772-df2a003c793d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:33.470740 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb982c39-b672-4821-9772-df2a003c793d service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:33.480418 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1046/2070] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:33.481202 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb982c39-b672-4821-9772-df2a003c793d service cinder] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:33.492618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45223686-ae99-4321-8b41-8a9215b2493f admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:33.493344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb982c39-b672-4821-9772-df2a003c793d service cinder] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:33.497677 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c9668-95ac-4ffe-ab23-a093ec69bf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:33.498292 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c9668-95ac-4ffe-ab23-a093ec69bf3a admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.498634 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c9668-95ac-4ffe-ab23-a093ec69bf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.499040 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c9668-95ac-4ffe-ab23-a093ec69bf3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.499518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c9668-95ac-4ffe-ab23-a093ec69bf3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:33.500708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0f79c-b45a-46f8-b67b-2dcc7cebcda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:33.501391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0f79c-b45a-46f8-b67b-2dcc7cebcda4 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.501491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c9668-95ac-4ffe-ab23-a093ec69bf3a admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:33.501805 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0f79c-b45a-46f8-b67b-2dcc7cebcda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.502138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0f79c-b45a-46f8-b67b-2dcc7cebcda4 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.502829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab0f79c-b45a-46f8-b67b-2dcc7cebcda4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:33.504950 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab0f79c-b45a-46f8-b67b-2dcc7cebcda4 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:33.533538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1047/2071] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:33 2022] POST /identity/v3/users => generated 299 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:33.533704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1025/2072] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:33 2022] POST /identity/v3/projects => generated 361 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:33.535270 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0f79c-b45a-46f8-b67b-2dcc7cebcda4 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:33.537551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c9668-95ac-4ffe-ab23-a093ec69bf3a admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:33.548181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0f79c-b45a-46f8-b67b-2dcc7cebcda4 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:33.615755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c9668-95ac-4ffe-ab23-a093ec69bf3a admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:33.619553 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2887ec9-5434-4140-9ee0-3d0721c796c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:33.620191 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2887ec9-5434-4140-9ee0-3d0721c796c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.621106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2887ec9-5434-4140-9ee0-3d0721c796c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.621391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2887ec9-5434-4140-9ee0-3d0721c796c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.622203 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2887ec9-5434-4140-9ee0-3d0721c796c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:33.622322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802df32-e354-4bb6-8e2c-6806ef5c3a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:33.622960 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802df32-e354-4bb6-8e2c-6806ef5c3a7e admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.623300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802df32-e354-4bb6-8e2c-6806ef5c3a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.623718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802df32-e354-4bb6-8e2c-6806ef5c3a7e admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.624071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2887ec9-5434-4140-9ee0-3d0721c796c3 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:33.624554 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3802df32-e354-4bb6-8e2c-6806ef5c3a7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:33.626978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3802df32-e354-4bb6-8e2c-6806ef5c3a7e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:33.640455 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1048/2073] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:33.641878 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2887ec9-5434-4140-9ee0-3d0721c796c3 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:33.658758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2887ec9-5434-4140-9ee0-3d0721c796c3 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:33.660971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1026/2074] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:33 2022] POST /identity/v3/users => generated 281 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:33.661464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802df32-e354-4bb6-8e2c-6806ef5c3a7e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:33.666728 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd26562-7699-425d-8a3b-cea6d451936d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:33.667773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd26562-7699-425d-8a3b-cea6d451936d admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.668040 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd26562-7699-425d-8a3b-cea6d451936d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.668390 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd26562-7699-425d-8a3b-cea6d451936d admin admin] PATH_INFO: `/v3/projects/4cbfac92f8ef46af83fe3a96434870cc/users/bf031e28f7d84f2b80f8f4940ccf1889/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.672326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802df32-e354-4bb6-8e2c-6806ef5c3a7e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:33.679626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f05f3-358c-4bc9-b659-466c927fab80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:33.680380 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f05f3-358c-4bc9-b659-466c927fab80 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.680732 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f05f3-358c-4bc9-b659-466c927fab80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.686966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f05f3-358c-4bc9-b659-466c927fab80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.686966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f05f3-358c-4bc9-b659-466c927fab80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:33.686966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f05f3-358c-4bc9-b659-466c927fab80 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:33.697381 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1027/2075] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:33.700045 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd26562-7699-425d-8a3b-cea6d451936d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cbfac92f8ef46af83fe3a96434870cc, user_id=bf031e28f7d84f2b80f8f4940ccf1889, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:33.701636 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f05f3-358c-4bc9-b659-466c927fab80 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:33.702966 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd26562-7699-425d-8a3b-cea6d451936d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:33.714492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f05f3-358c-4bc9-b659-466c927fab80 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:33.720753 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1049/2076] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:33 2022] PUT /identity/v3/projects/4cbfac92f8ef46af83fe3a96434870cc/users/bf031e28f7d84f2b80f8f4940ccf1889/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:33.726221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd26562-7699-425d-8a3b-cea6d451936d admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:33.794865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:33.796145 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc32b5-6dd1-4c0c-81d6-b40820daae3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:33.797070 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc32b5-6dd1-4c0c-81d6-b40820daae3f admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.797499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc32b5-6dd1-4c0c-81d6-b40820daae3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.797896 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc32b5-6dd1-4c0c-81d6-b40820daae3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.798829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbc32b5-6dd1-4c0c-81d6-b40820daae3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:33.800640 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbc32b5-6dd1-4c0c-81d6-b40820daae3f admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:33.813040 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1028/2077] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:33.817857 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc32b5-6dd1-4c0c-81d6-b40820daae3f admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:33.839653 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd26562-7699-425d-8a3b-cea6d451936d admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:33.839764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc32b5-6dd1-4c0c-81d6-b40820daae3f admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:33.845063 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb16413-93e3-428b-9cec-3db1cc66c8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:33.845827 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb16413-93e3-428b-9cec-3db1cc66c8fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.846173 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb16413-93e3-428b-9cec-3db1cc66c8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.846551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb16413-93e3-428b-9cec-3db1cc66c8fc admin admin] PATH_INFO: `/v3/projects/6d1766122d51424f9c1a7a938ccd98bc/users/e4342ae329a04222b700d472b10b2674/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.849352 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a389087-cacb-4d22-9a74-e7677efae697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:33.857588 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a389087-cacb-4d22-9a74-e7677efae697 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.857936 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a389087-cacb-4d22-9a74-e7677efae697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.858217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a389087-cacb-4d22-9a74-e7677efae697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.859447 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a389087-cacb-4d22-9a74-e7677efae697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:33.861392 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a389087-cacb-4d22-9a74-e7677efae697 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:33.878076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb16413-93e3-428b-9cec-3db1cc66c8fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d1766122d51424f9c1a7a938ccd98bc, user_id=e4342ae329a04222b700d472b10b2674, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:33.878076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb16413-93e3-428b-9cec-3db1cc66c8fc admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:33.883376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1050/2078] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:33.888277 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a389087-cacb-4d22-9a74-e7677efae697 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:33.891976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:33.903559 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1029/2079] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:33 2022] PUT /identity/v3/projects/6d1766122d51424f9c1a7a938ccd98bc/users/e4342ae329a04222b700d472b10b2674/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:33.906497 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a389087-cacb-4d22-9a74-e7677efae697 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:33.910265 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ad20b-934d-4c8c-bfa5-f34d369678c6 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.910621 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ad20b-934d-4c8c-bfa5-f34d369678c6 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.910954 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ad20b-934d-4c8c-bfa5-f34d369678c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.947971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-1f0ad20b-934d-4c8c-bfa5-f34d369678c6 None None] MFA Rules not processed for user `e4342ae329a04222b700d472b10b2674`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:33.948824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-1f0ad20b-934d-4c8c-bfa5-f34d369678c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:33.976207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae3f91-c598-4a9b-8106-844d9f189f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:33.976954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae3f91-c598-4a9b-8106-844d9f189f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:33.977298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae3f91-c598-4a9b-8106-844d9f189f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:33.977642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae3f91-c598-4a9b-8106-844d9f189f11 admin admin] PATH_INFO: `/v3/projects/4cbfac92f8ef46af83fe3a96434870cc/users/bf031e28f7d84f2b80f8f4940ccf1889/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:33.979886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae3f91-c598-4a9b-8106-844d9f189f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cbfac92f8ef46af83fe3a96434870cc, user_id=bf031e28f7d84f2b80f8f4940ccf1889, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:33.982640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae3f91-c598-4a9b-8106-844d9f189f11 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:33.994239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1051/2080] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:33 2022] PUT /identity/v3/projects/4cbfac92f8ef46af83fe3a96434870cc/users/bf031e28f7d84f2b80f8f4940ccf1889/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:34.006569 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5aa80-eecb-420e-bc22-dac408c825ad None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:34.006793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5aa80-eecb-420e-bc22-dac408c825ad None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:34.007011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5aa80-eecb-420e-bc22-dac408c825ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:34.022260 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1030/2081] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:34.043789 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-c7a5aa80-eecb-420e-bc22-dac408c825ad None None] MFA Rules not processed for user `bf031e28f7d84f2b80f8f4940ccf1889`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:34.046145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c7a5aa80-eecb-420e-bc22-dac408c825ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:34.058194 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ad20b-934d-4c8c-bfa5-f34d369678c6 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:34.112902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ad20b-934d-4c8c-bfa5-f34d369678c6 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:34.125852 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1052/2082] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:34.133626 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5aa80-eecb-420e-bc22-dac408c825ad None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:34.135746 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c7a5aa80-eecb-420e-bc22-dac408c825ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:34.163387 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688c888-92d3-405e-85f9-e9e9202f6ddd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:34.164016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688c888-92d3-405e-85f9-e9e9202f6ddd service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:34.164293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688c888-92d3-405e-85f9-e9e9202f6ddd service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:34.164565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688c888-92d3-405e-85f9-e9e9202f6ddd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:34.195185 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5aa80-eecb-420e-bc22-dac408c825ad None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:34.199469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31611c0-b65d-41df-939a-341790e2753c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:34.200079 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31611c0-b65d-41df-939a-341790e2753c service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:34.200354 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31611c0-b65d-41df-939a-341790e2753c service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:34.200623 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31611c0-b65d-41df-939a-341790e2753c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:34.204794 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:34.221859 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0688c888-92d3-405e-85f9-e9e9202f6ddd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:34.223330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0688c888-92d3-405e-85f9-e9e9202f6ddd service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:34.229156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:34.234417 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1031/2083] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:13:34.250045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688c888-92d3-405e-85f9-e9e9202f6ddd service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:34.260802 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31611c0-b65d-41df-939a-341790e2753c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:34.262262 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31611c0-b65d-41df-939a-341790e2753c service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:34.271419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1053/2084] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:13:34.314426 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688c888-92d3-405e-85f9-e9e9202f6ddd service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:34.319210 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf91d41-c045-4e05-b2ac-88411d7e9fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:34.319882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf91d41-c045-4e05-b2ac-88411d7e9fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:34.320197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf91d41-c045-4e05-b2ac-88411d7e9fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:34.320468 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf91d41-c045-4e05-b2ac-88411d7e9fa7 admin admin] PATH_INFO: `/v3/users/e4342ae329a04222b700d472b10b2674` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:34.321685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf91d41-c045-4e05-b2ac-88411d7e9fa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4342ae329a04222b700d472b10b2674)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:34.323683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf91d41-c045-4e05-b2ac-88411d7e9fa7 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:34.380430 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-6cf91d41-c045-4e05-b2ac-88411d7e9fa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4342ae329a04222b700d472b10b2674'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:34.381263 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-6cf91d41-c045-4e05-b2ac-88411d7e9fa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4342ae329a04222b700d472b10b2674'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:34.381984 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-6cf91d41-c045-4e05-b2ac-88411d7e9fa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4342ae329a04222b700d472b10b2674'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:34.389901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-6cf91d41-c045-4e05-b2ac-88411d7e9fa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4342ae329a04222b700d472b10b2674'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:34.411092 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-6cf91d41-c045-4e05-b2ac-88411d7e9fa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4342ae329a04222b700d472b10b2674'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:34.429053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1032/2085] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:34 2022] DELETE /identity/v3/users/e4342ae329a04222b700d472b10b2674 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:34.434412 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e25713-470b-4e8d-941c-495760d22c84 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:34.434884 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e25713-470b-4e8d-941c-495760d22c84 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:34.435144 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e25713-470b-4e8d-941c-495760d22c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:34.455356 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-74e25713-470b-4e8d-941c-495760d22c84 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:34.456518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-74e25713-470b-4e8d-941c-495760d22c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:34.525251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1054/2086] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:34.536319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf91d41-c045-4e05-b2ac-88411d7e9fa7 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:34.537736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-6cf91d41-c045-4e05-b2ac-88411d7e9fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:34.601385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf91d41-c045-4e05-b2ac-88411d7e9fa7 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:34.605194 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e1331-fc1c-4c28-bc8d-daf5d7d985be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:34.606096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e1331-fc1c-4c28-bc8d-daf5d7d985be service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:34.606276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e1331-fc1c-4c28-bc8d-daf5d7d985be service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:34.606631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e1331-fc1c-4c28-bc8d-daf5d7d985be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:34.617124 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e1331-fc1c-4c28-bc8d-daf5d7d985be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:34.618796 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e1331-fc1c-4c28-bc8d-daf5d7d985be service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:34.629102 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1033/2087] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:34.946115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e25713-470b-4e8d-941c-495760d22c84 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:34.948144 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-74e25713-470b-4e8d-941c-495760d22c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:34.969884 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e25713-470b-4e8d-941c-495760d22c84 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:34.986033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3c36d-08a3-463c-9885-dd7c044dd77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:34.986939 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3c36d-08a3-463c-9885-dd7c044dd77c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:34.987165 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3c36d-08a3-463c-9885-dd7c044dd77c admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:34.987663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3c36d-08a3-463c-9885-dd7c044dd77c admin admin] PATH_INFO: `/v3/projects/6d1766122d51424f9c1a7a938ccd98bc` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:34.988933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3c36d-08a3-463c-9885-dd7c044dd77c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d1766122d51424f9c1a7a938ccd98bc)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:34.990906 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3c36d-08a3-463c-9885-dd7c044dd77c admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:35.048571 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-bbc3c36d-08a3-463c-9885-dd7c044dd77c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d1766122d51424f9c1a7a938ccd98bc', 'request_id': 'req-bbc3c36d-08a3-463c-9885-dd7c044dd77c'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:35.054468 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-bbc3c36d-08a3-463c-9885-dd7c044dd77c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d1766122d51424f9c1a7a938ccd98bc', 'request_id': 'req-bbc3c36d-08a3-463c-9885-dd7c044dd77c'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:35.064061 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-bbc3c36d-08a3-463c-9885-dd7c044dd77c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d1766122d51424f9c1a7a938ccd98bc', 'request_id': 'req-bbc3c36d-08a3-463c-9885-dd7c044dd77c'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:35.079587 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-bbc3c36d-08a3-463c-9885-dd7c044dd77c admin admin] The token cache is being invalidate because project 6d1766122d51424f9c1a7a938ccd98bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:35.079911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-bbc3c36d-08a3-463c-9885-dd7c044dd77c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:35.080683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-bbc3c36d-08a3-463c-9885-dd7c044dd77c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:35.086869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1055/2088] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:34 2022] DELETE /identity/v3/projects/6d1766122d51424f9c1a7a938ccd98bc => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:35.090796 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e440ba1-2c8f-4d71-8a91-96812b2aa6e0 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:35.091071 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e440ba1-2c8f-4d71-8a91-96812b2aa6e0 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:35.091354 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e440ba1-2c8f-4d71-8a91-96812b2aa6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:35.111059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-2e440ba1-2c8f-4d71-8a91-96812b2aa6e0 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:35.112313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2e440ba1-2c8f-4d71-8a91-96812b2aa6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:35.131484 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:35.180941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1034/2089] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:35.189950 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74def28-1109-450f-a1ce-3c8407929374 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:35.190252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74def28-1109-450f-a1ce-3c8407929374 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:35.190522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74def28-1109-450f-a1ce-3c8407929374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:35.212525 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-c74def28-1109-450f-a1ce-3c8407929374 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:35.213745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c74def28-1109-450f-a1ce-3c8407929374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:35.222914 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1056/2090] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:35.227157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e440ba1-2c8f-4d71-8a91-96812b2aa6e0 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:35.237257 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e440ba1-2c8f-4d71-8a91-96812b2aa6e0 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:35.242137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd489e4f-4f5c-4959-a90b-556e970eef90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:35.242738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd489e4f-4f5c-4959-a90b-556e970eef90 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:35.243012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd489e4f-4f5c-4959-a90b-556e970eef90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:35.243280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd489e4f-4f5c-4959-a90b-556e970eef90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:35.244199 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd489e4f-4f5c-4959-a90b-556e970eef90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:35.244501 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd489e4f-4f5c-4959-a90b-556e970eef90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:35.246125 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd489e4f-4f5c-4959-a90b-556e970eef90 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:35.257984 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1035/2091] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:35.262418 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74def28-1109-450f-a1ce-3c8407929374 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:35.272424 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74def28-1109-450f-a1ce-3c8407929374 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:35.277903 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876db441-d12a-4513-a804-1445939ce3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:35.278501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876db441-d12a-4513-a804-1445939ce3f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:35.278781 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876db441-d12a-4513-a804-1445939ce3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:35.279052 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876db441-d12a-4513-a804-1445939ce3f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:35.279723 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876db441-d12a-4513-a804-1445939ce3f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:35.281639 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876db441-d12a-4513-a804-1445939ce3f9 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:35.306324 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:35.317056 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1057/2092] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:35 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:35.322752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd489e4f-4f5c-4959-a90b-556e970eef90 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:35.333436 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd489e4f-4f5c-4959-a90b-556e970eef90 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:35.392253 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327002b-b367-479c-bd33-756e164af104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:35.392916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327002b-b367-479c-bd33-756e164af104 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:35.393135 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327002b-b367-479c-bd33-756e164af104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:35.393410 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327002b-b367-479c-bd33-756e164af104 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:35.394058 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f327002b-b367-479c-bd33-756e164af104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:35.396076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f327002b-b367-479c-bd33-756e164af104 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:35.421529 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1036/2093] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:35 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:35.422765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876db441-d12a-4513-a804-1445939ce3f9 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:35.433570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876db441-d12a-4513-a804-1445939ce3f9 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:35.439292 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07109cc8-0907-424f-ab46-5a0268191b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:35.440561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07109cc8-0907-424f-ab46-5a0268191b77 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:35.440909 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07109cc8-0907-424f-ab46-5a0268191b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:35.441119 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07109cc8-0907-424f-ab46-5a0268191b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:35.441776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07109cc8-0907-424f-ab46-5a0268191b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:35.443387 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07109cc8-0907-424f-ab46-5a0268191b77 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:35.456202 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1058/2094] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:35.460587 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327002b-b367-479c-bd33-756e164af104 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:35.479869 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327002b-b367-479c-bd33-756e164af104 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:35.484720 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff9b62-3a50-451d-9772-60f6db64adef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:35.485334 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff9b62-3a50-451d-9772-60f6db64adef admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:35.485635 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff9b62-3a50-451d-9772-60f6db64adef admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:35.485909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff9b62-3a50-451d-9772-60f6db64adef admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:35.486672 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff9b62-3a50-451d-9772-60f6db64adef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:35.488403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff9b62-3a50-451d-9772-60f6db64adef admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:35.500378 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1037/2095] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:35.504978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07109cc8-0907-424f-ab46-5a0268191b77 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:35.516224 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07109cc8-0907-424f-ab46-5a0268191b77 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:35.521331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c25b66-0d89-4d27-9909-efc9e2d14176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:35.521989 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c25b66-0d89-4d27-9909-efc9e2d14176 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:35.522263 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c25b66-0d89-4d27-9909-efc9e2d14176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:35.522565 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c25b66-0d89-4d27-9909-efc9e2d14176 admin admin] PATH_INFO: `/v3/projects/0990a154c5ef407d867435c4fc3f599f/users/7a84f1bef124445baa45ae4f2d8c3f4c/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:35.542896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c25b66-0d89-4d27-9909-efc9e2d14176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0990a154c5ef407d867435c4fc3f599f, user_id=7a84f1bef124445baa45ae4f2d8c3f4c, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:35.545817 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c25b66-0d89-4d27-9909-efc9e2d14176 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:35.559561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1059/2096] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:35 2022] PUT /identity/v3/projects/0990a154c5ef407d867435c4fc3f599f/users/7a84f1bef124445baa45ae4f2d8c3f4c/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:35.569976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd801cc7-78b7-4b9c-b7f9-7448a5ad52cb None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:35.570289 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd801cc7-78b7-4b9c-b7f9-7448a5ad52cb None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:35.571167 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd801cc7-78b7-4b9c-b7f9-7448a5ad52cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:35.599808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-fd801cc7-78b7-4b9c-b7f9-7448a5ad52cb None None] MFA Rules not processed for user `7a84f1bef124445baa45ae4f2d8c3f4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:35.601029 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-fd801cc7-78b7-4b9c-b7f9-7448a5ad52cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:35.673591 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1038/2097] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:35.686579 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c25b66-0d89-4d27-9909-efc9e2d14176 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:35.688207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-f3c25b66-0d89-4d27-9909-efc9e2d14176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:35.755536 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c25b66-0d89-4d27-9909-efc9e2d14176 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:35.773496 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815d0ee-75b0-4c29-a1d0-7db4ae984fe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:35.774690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815d0ee-75b0-4c29-a1d0-7db4ae984fe1 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:35.775039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815d0ee-75b0-4c29-a1d0-7db4ae984fe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:35.775311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815d0ee-75b0-4c29-a1d0-7db4ae984fe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:35.785205 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815d0ee-75b0-4c29-a1d0-7db4ae984fe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:35.786666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815d0ee-75b0-4c29-a1d0-7db4ae984fe1 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:35.796493 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1060/2098] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:41.799439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd801cc7-78b7-4b9c-b7f9-7448a5ad52cb None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:41.819303 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:41.865342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd801cc7-78b7-4b9c-b7f9-7448a5ad52cb None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:41.870537 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6cfe2-02de-4110-aeb8-7978834b5616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:41.871040 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6cfe2-02de-4110-aeb8-7978834b5616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:41.871331 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6cfe2-02de-4110-aeb8-7978834b5616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:41.871638 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6cfe2-02de-4110-aeb8-7978834b5616 admin admin] PATH_INFO: `/v3/users/7a84f1bef124445baa45ae4f2d8c3f4c` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:41.872778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6cfe2-02de-4110-aeb8-7978834b5616 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a84f1bef124445baa45ae4f2d8c3f4c)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:41.874768 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6cfe2-02de-4110-aeb8-7978834b5616 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:41.931667 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b5a6cfe2-02de-4110-aeb8-7978834b5616 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a84f1bef124445baa45ae4f2d8c3f4c'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:41.932530 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b5a6cfe2-02de-4110-aeb8-7978834b5616 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a84f1bef124445baa45ae4f2d8c3f4c'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:41.933774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b5a6cfe2-02de-4110-aeb8-7978834b5616 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a84f1bef124445baa45ae4f2d8c3f4c'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:41.942732 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b5a6cfe2-02de-4110-aeb8-7978834b5616 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a84f1bef124445baa45ae4f2d8c3f4c'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:41.960199 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b5a6cfe2-02de-4110-aeb8-7978834b5616 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a84f1bef124445baa45ae4f2d8c3f4c'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:41.987659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1039/2099] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:41 2022] DELETE /identity/v3/users/7a84f1bef124445baa45ae4f2d8c3f4c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:41.989548 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227c31b-5e9d-4670-b7b6-f8fc49f842d0 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:41.990026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227c31b-5e9d-4670-b7b6-f8fc49f842d0 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:41.990199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227c31b-5e9d-4670-b7b6-f8fc49f842d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:42.017607 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-2227c31b-5e9d-4670-b7b6-f8fc49f842d0 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:42.019031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-2227c31b-5e9d-4670-b7b6-f8fc49f842d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:42.094564 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1061/2100] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:42.104681 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6cfe2-02de-4110-aeb8-7978834b5616 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:42.106589 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b5a6cfe2-02de-4110-aeb8-7978834b5616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:42.147243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227c31b-5e9d-4670-b7b6-f8fc49f842d0 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:42.147243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-2227c31b-5e9d-4670-b7b6-f8fc49f842d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:42.164748 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227c31b-5e9d-4670-b7b6-f8fc49f842d0 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:42.171970 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e839b-41bb-419b-b4b0-84c654129f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:42.172971 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e839b-41bb-419b-b4b0-84c654129f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:42.173543 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e839b-41bb-419b-b4b0-84c654129f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:42.174124 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e839b-41bb-419b-b4b0-84c654129f08 admin admin] PATH_INFO: `/v3/users/c6a6dae1083b4729b237dcf83c75a550` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:42.177284 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e839b-41bb-419b-b4b0-84c654129f08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6a6dae1083b4729b237dcf83c75a550)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:42.186647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e839b-41bb-419b-b4b0-84c654129f08 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:42.234579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6cfe2-02de-4110-aeb8-7978834b5616 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:42.241183 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953160af-3487-40aa-8b71-8d3d466def86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:42.242106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953160af-3487-40aa-8b71-8d3d466def86 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:42.242799 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953160af-3487-40aa-8b71-8d3d466def86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:42.243298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953160af-3487-40aa-8b71-8d3d466def86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:42.255850 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953160af-3487-40aa-8b71-8d3d466def86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:42.257519 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953160af-3487-40aa-8b71-8d3d466def86 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:42.269897 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1040/2101] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:42.278934 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953160af-3487-40aa-8b71-8d3d466def86 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:42.280888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-953160af-3487-40aa-8b71-8d3d466def86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:42.302560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-430e839b-41bb-419b-b4b0-84c654129f08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6a6dae1083b4729b237dcf83c75a550'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:42.304258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-430e839b-41bb-419b-b4b0-84c654129f08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6a6dae1083b4729b237dcf83c75a550'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:42.305010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-430e839b-41bb-419b-b4b0-84c654129f08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6a6dae1083b4729b237dcf83c75a550'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:42.323049 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-430e839b-41bb-419b-b4b0-84c654129f08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6a6dae1083b4729b237dcf83c75a550'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:42.337545 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:42.342343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:42.353860 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-430e839b-41bb-419b-b4b0-84c654129f08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6a6dae1083b4729b237dcf83c75a550'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:42.369178 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953160af-3487-40aa-8b71-8d3d466def86 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:42.373857 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686b533-3925-471e-958d-35c8ac484aa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:42.374649 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686b533-3925-471e-958d-35c8ac484aa7 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:42.374928 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686b533-3925-471e-958d-35c8ac484aa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:42.375584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686b533-3925-471e-958d-35c8ac484aa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:42.378396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f686b533-3925-471e-958d-35c8ac484aa7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:42.388407 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1062/2102] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:42 2022] DELETE /identity/v3/users/c6a6dae1083b4729b237dcf83c75a550 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:42.395258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a302b3-a27d-4b31-bc8e-a088535346a4 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:42.395806 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a302b3-a27d-4b31-bc8e-a088535346a4 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:42.398578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a302b3-a27d-4b31-bc8e-a088535346a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:42.428623 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-d1a302b3-a27d-4b31-bc8e-a088535346a4 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:42.429950 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d1a302b3-a27d-4b31-bc8e-a088535346a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:42.506692 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686b533-3925-471e-958d-35c8ac484aa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:42.508892 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686b533-3925-471e-958d-35c8ac484aa7 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:42.521671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1063/2103] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:42.532987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a302b3-a27d-4b31-bc8e-a088535346a4 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:42.536631 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d1a302b3-a27d-4b31-bc8e-a088535346a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:42.555714 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1041/2104] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:42.631784 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a302b3-a27d-4b31-bc8e-a088535346a4 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:42.636082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9efef-6b1d-4ffb-bebc-8533a6f48271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:42.636967 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9efef-6b1d-4ffb-bebc-8533a6f48271 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:42.637319 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9efef-6b1d-4ffb-bebc-8533a6f48271 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:42.637954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9efef-6b1d-4ffb-bebc-8533a6f48271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:42.649460 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e9efef-6b1d-4ffb-bebc-8533a6f48271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:42.651354 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e9efef-6b1d-4ffb-bebc-8533a6f48271 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:42.664381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1064/2105] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:42.765832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686b533-3925-471e-958d-35c8ac484aa7 service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:42.768448 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f686b533-3925-471e-958d-35c8ac484aa7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:42.787589 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686b533-3925-471e-958d-35c8ac484aa7 service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:42.796227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e75a91-8539-4e32-b764-8eb2fcfb155f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:42.796950 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e75a91-8539-4e32-b764-8eb2fcfb155f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:42.797320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e75a91-8539-4e32-b764-8eb2fcfb155f admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:42.797724 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e75a91-8539-4e32-b764-8eb2fcfb155f admin admin] PATH_INFO: `/v3/projects/0990a154c5ef407d867435c4fc3f599f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:42.800315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e75a91-8539-4e32-b764-8eb2fcfb155f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0990a154c5ef407d867435c4fc3f599f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:42.803255 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e75a91-8539-4e32-b764-8eb2fcfb155f admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:42.875223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d5e75a91-8539-4e32-b764-8eb2fcfb155f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0990a154c5ef407d867435c4fc3f599f', 'request_id': 'req-d5e75a91-8539-4e32-b764-8eb2fcfb155f'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:42.882882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d5e75a91-8539-4e32-b764-8eb2fcfb155f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0990a154c5ef407d867435c4fc3f599f', 'request_id': 'req-d5e75a91-8539-4e32-b764-8eb2fcfb155f'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:42.893174 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d5e75a91-8539-4e32-b764-8eb2fcfb155f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0990a154c5ef407d867435c4fc3f599f', 'request_id': 'req-d5e75a91-8539-4e32-b764-8eb2fcfb155f'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:42.906035 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d5e75a91-8539-4e32-b764-8eb2fcfb155f admin admin] The token cache is being invalidate because project 0990a154c5ef407d867435c4fc3f599f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:42.906462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d5e75a91-8539-4e32-b764-8eb2fcfb155f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:42.907198 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d5e75a91-8539-4e32-b764-8eb2fcfb155f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:42.909562 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1042/2106] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:42 2022] DELETE /identity/v3/projects/0990a154c5ef407d867435c4fc3f599f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:43.015573 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9efef-6b1d-4ffb-bebc-8533a6f48271 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:43.018724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-83e9efef-6b1d-4ffb-bebc-8533a6f48271 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:43.051361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9efef-6b1d-4ffb-bebc-8533a6f48271 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:43.134997 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9aa39-b1a0-4d21-998e-6d870afeb51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:43.135871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9aa39-b1a0-4d21-998e-6d870afeb51a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:43.136237 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9aa39-b1a0-4d21-998e-6d870afeb51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:43.136516 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9aa39-b1a0-4d21-998e-6d870afeb51a admin admin] PATH_INFO: `/v3/projects/4a73dcb1b32b4d8597b76cb0b27aefcf` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:43.137702 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9aa39-b1a0-4d21-998e-6d870afeb51a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a73dcb1b32b4d8597b76cb0b27aefcf)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:43.139663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9aa39-b1a0-4d21-998e-6d870afeb51a admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:43.173673 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:43.192942 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-0af9aa39-b1a0-4d21-998e-6d870afeb51a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a73dcb1b32b4d8597b76cb0b27aefcf', 'request_id': 'req-0af9aa39-b1a0-4d21-998e-6d870afeb51a'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:43.198654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-0af9aa39-b1a0-4d21-998e-6d870afeb51a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a73dcb1b32b4d8597b76cb0b27aefcf', 'request_id': 'req-0af9aa39-b1a0-4d21-998e-6d870afeb51a'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:43.208032 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-0af9aa39-b1a0-4d21-998e-6d870afeb51a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a73dcb1b32b4d8597b76cb0b27aefcf', 'request_id': 'req-0af9aa39-b1a0-4d21-998e-6d870afeb51a'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:43.219480 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-0af9aa39-b1a0-4d21-998e-6d870afeb51a admin admin] The token cache is being invalidate because project 4a73dcb1b32b4d8597b76cb0b27aefcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:43.219851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-0af9aa39-b1a0-4d21-998e-6d870afeb51a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:43.220475 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-0af9aa39-b1a0-4d21-998e-6d870afeb51a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:43.221974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1065/2107] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:43 2022] DELETE /identity/v3/projects/4a73dcb1b32b4d8597b76cb0b27aefcf => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:43.226006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e75a91-8539-4e32-b764-8eb2fcfb155f admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:43.227624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d5e75a91-8539-4e32-b764-8eb2fcfb155f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:43.240659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e75a91-8539-4e32-b764-8eb2fcfb155f admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:43.301723 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6e853-ada0-47ec-9d36-d379c3cb026a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:43.302992 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6e853-ada0-47ec-9d36-d379c3cb026a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:43.303270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6e853-ada0-47ec-9d36-d379c3cb026a admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:43.303583 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6e853-ada0-47ec-9d36-d379c3cb026a admin admin] PATH_INFO: `/v3/users/053a8ad2ad004dbeb09f66c6273c8f05` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:43.304717 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6e853-ada0-47ec-9d36-d379c3cb026a admin admin] RBAC: Authorizing `identity:delete_user(user_id=053a8ad2ad004dbeb09f66c6273c8f05)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:43.306782 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6e853-ada0-47ec-9d36-d379c3cb026a admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:43.321789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:43.358375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-bfe6e853-ada0-47ec-9d36-d379c3cb026a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '053a8ad2ad004dbeb09f66c6273c8f05'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:43.359027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-bfe6e853-ada0-47ec-9d36-d379c3cb026a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '053a8ad2ad004dbeb09f66c6273c8f05'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:43.359659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-bfe6e853-ada0-47ec-9d36-d379c3cb026a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '053a8ad2ad004dbeb09f66c6273c8f05'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:43.366659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-bfe6e853-ada0-47ec-9d36-d379c3cb026a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '053a8ad2ad004dbeb09f66c6273c8f05'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:43.378682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-bfe6e853-ada0-47ec-9d36-d379c3cb026a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '053a8ad2ad004dbeb09f66c6273c8f05'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:43.394899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1043/2108] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:43 2022] DELETE /identity/v3/users/053a8ad2ad004dbeb09f66c6273c8f05 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:43.703312 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9aa39-b1a0-4d21-998e-6d870afeb51a admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:43.707278 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-0af9aa39-b1a0-4d21-998e-6d870afeb51a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:43.725121 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9aa39-b1a0-4d21-998e-6d870afeb51a admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:43.791401 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394eab9b-aa46-49a7-b6d2-9064b78a3ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:43.792074 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394eab9b-aa46-49a7-b6d2-9064b78a3ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:43.792347 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394eab9b-aa46-49a7-b6d2-9064b78a3ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:43.792619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394eab9b-aa46-49a7-b6d2-9064b78a3ad8 admin admin] PATH_INFO: `/v3/projects/6972c61a598e4a798aa9c5832256108d` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:43.793848 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394eab9b-aa46-49a7-b6d2-9064b78a3ad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6972c61a598e4a798aa9c5832256108d)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:43.795833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394eab9b-aa46-49a7-b6d2-9064b78a3ad8 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:43.851439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-394eab9b-aa46-49a7-b6d2-9064b78a3ad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6972c61a598e4a798aa9c5832256108d', 'request_id': 'req-394eab9b-aa46-49a7-b6d2-9064b78a3ad8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:43.857056 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-394eab9b-aa46-49a7-b6d2-9064b78a3ad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6972c61a598e4a798aa9c5832256108d', 'request_id': 'req-394eab9b-aa46-49a7-b6d2-9064b78a3ad8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:43.864994 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-394eab9b-aa46-49a7-b6d2-9064b78a3ad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6972c61a598e4a798aa9c5832256108d', 'request_id': 'req-394eab9b-aa46-49a7-b6d2-9064b78a3ad8'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:43.876746 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-394eab9b-aa46-49a7-b6d2-9064b78a3ad8 admin admin] The token cache is being invalidate because project 6972c61a598e4a798aa9c5832256108d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:43.877031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-394eab9b-aa46-49a7-b6d2-9064b78a3ad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:43.877697 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-394eab9b-aa46-49a7-b6d2-9064b78a3ad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:43.879250 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1066/2109] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:43 2022] DELETE /identity/v3/projects/6972c61a598e4a798aa9c5832256108d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:43.884317 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3edb6a-5308-4d24-8163-7d9309c26d88 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:43.884798 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3edb6a-5308-4d24-8163-7d9309c26d88 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:43.884924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3edb6a-5308-4d24-8163-7d9309c26d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:43.905056 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-0f3edb6a-5308-4d24-8163-7d9309c26d88 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:43.906275 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-0f3edb6a-5308-4d24-8163-7d9309c26d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:43.966178 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1044/2110] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:43.974533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72671e-93f2-4544-ac49-635eb731ca06 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:43.974824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72671e-93f2-4544-ac49-635eb731ca06 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:43.975127 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72671e-93f2-4544-ac49-635eb731ca06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:43.996651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-ce72671e-93f2-4544-ac49-635eb731ca06 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:43.997887 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ce72671e-93f2-4544-ac49-635eb731ca06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:44.007880 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1067/2111] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:44.012406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3edb6a-5308-4d24-8163-7d9309c26d88 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:44.022326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3edb6a-5308-4d24-8163-7d9309c26d88 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:44.027321 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2511eda-81f9-4d5c-9810-8ddca7c7d42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:44.027971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2511eda-81f9-4d5c-9810-8ddca7c7d42e admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:44.028223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2511eda-81f9-4d5c-9810-8ddca7c7d42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:44.028516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2511eda-81f9-4d5c-9810-8ddca7c7d42e admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:44.029159 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2511eda-81f9-4d5c-9810-8ddca7c7d42e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:44.029457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2511eda-81f9-4d5c-9810-8ddca7c7d42e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:44.031077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2511eda-81f9-4d5c-9810-8ddca7c7d42e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:44.042756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1045/2112] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:44.047216 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72671e-93f2-4544-ac49-635eb731ca06 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:44.056878 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72671e-93f2-4544-ac49-635eb731ca06 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:44.062117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ed9eb-51f7-48b7-9829-3d3a41d12372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:44.062716 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ed9eb-51f7-48b7-9829-3d3a41d12372 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:44.063033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ed9eb-51f7-48b7-9829-3d3a41d12372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:44.063248 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ed9eb-51f7-48b7-9829-3d3a41d12372 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:44.064025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ed9eb-51f7-48b7-9829-3d3a41d12372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:44.065899 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ed9eb-51f7-48b7-9829-3d3a41d12372 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:44.098953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1068/2113] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:44 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:44.104701 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2511eda-81f9-4d5c-9810-8ddca7c7d42e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:44.115299 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2511eda-81f9-4d5c-9810-8ddca7c7d42e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:44.181571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4fc35-609e-45a6-8658-34fe8e5cd124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:44.182190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4fc35-609e-45a6-8658-34fe8e5cd124 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:44.182466 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4fc35-609e-45a6-8658-34fe8e5cd124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:44.182721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4fc35-609e-45a6-8658-34fe8e5cd124 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:44.183354 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d4fc35-609e-45a6-8658-34fe8e5cd124 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:44.185369 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d4fc35-609e-45a6-8658-34fe8e5cd124 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:44.210283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1046/2114] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:44 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:44.214624 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ed9eb-51f7-48b7-9829-3d3a41d12372 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:44.225748 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ed9eb-51f7-48b7-9829-3d3a41d12372 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:44.231301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf33c7a-f77d-4273-aaee-8c7c2ebb5ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:44.231941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf33c7a-f77d-4273-aaee-8c7c2ebb5ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:44.232210 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf33c7a-f77d-4273-aaee-8c7c2ebb5ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:44.232481 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf33c7a-f77d-4273-aaee-8c7c2ebb5ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:44.233126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf33c7a-f77d-4273-aaee-8c7c2ebb5ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:44.234727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf33c7a-f77d-4273-aaee-8c7c2ebb5ab7 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:44.245434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1069/2115] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:44.249548 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4fc35-609e-45a6-8658-34fe8e5cd124 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:44.264763 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4fc35-609e-45a6-8658-34fe8e5cd124 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:44.271282 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be69a5f-c99b-4ade-8725-f4aa83511fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:44.271920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be69a5f-c99b-4ade-8725-f4aa83511fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:44.272199 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be69a5f-c99b-4ade-8725-f4aa83511fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:44.272468 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be69a5f-c99b-4ade-8725-f4aa83511fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:44.273106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be69a5f-c99b-4ade-8725-f4aa83511fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:44.274741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be69a5f-c99b-4ade-8725-f4aa83511fe9 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:44.277047 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:44.289595 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1047/2116] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:44.291646 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf33c7a-f77d-4273-aaee-8c7c2ebb5ab7 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:44.298212 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:44.302666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf33c7a-f77d-4273-aaee-8c7c2ebb5ab7 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:44.309715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aedc2d-dc1b-4a36-8a7b-50495d892994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:44.310290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aedc2d-dc1b-4a36-8a7b-50495d892994 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:44.310778 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aedc2d-dc1b-4a36-8a7b-50495d892994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:44.311225 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aedc2d-dc1b-4a36-8a7b-50495d892994 admin admin] PATH_INFO: `/v3/projects/47fe5a2568ba483891d6200556f1f4ec/users/4fb1d748076d44b7a5cfee53ed06c29e/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:44.327935 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aedc2d-dc1b-4a36-8a7b-50495d892994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47fe5a2568ba483891d6200556f1f4ec, user_id=4fb1d748076d44b7a5cfee53ed06c29e, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:44.330729 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aedc2d-dc1b-4a36-8a7b-50495d892994 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:44.345449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1070/2117] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:44 2022] PUT /identity/v3/projects/47fe5a2568ba483891d6200556f1f4ec/users/4fb1d748076d44b7a5cfee53ed06c29e/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:44.356053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6acbc-1d52-47d4-91bd-184670af3285 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:44.356338 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6acbc-1d52-47d4-91bd-184670af3285 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:44.356663 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6acbc-1d52-47d4-91bd-184670af3285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:44.385779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-5fa6acbc-1d52-47d4-91bd-184670af3285 None None] MFA Rules not processed for user `4fb1d748076d44b7a5cfee53ed06c29e`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:44.387000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-5fa6acbc-1d52-47d4-91bd-184670af3285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:44.452349 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1048/2118] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:44.464024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aedc2d-dc1b-4a36-8a7b-50495d892994 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:44.465849 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-15aedc2d-dc1b-4a36-8a7b-50495d892994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:44.525401 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aedc2d-dc1b-4a36-8a7b-50495d892994 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:44.529870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f69fc-6dee-4ee4-9c47-510a6d45e55a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:44.530461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f69fc-6dee-4ee4-9c47-510a6d45e55a service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:44.530735 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f69fc-6dee-4ee4-9c47-510a6d45e55a service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:44.531000 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f69fc-6dee-4ee4-9c47-510a6d45e55a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:44.539740 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f69fc-6dee-4ee4-9c47-510a6d45e55a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:44.541163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f69fc-6dee-4ee4-9c47-510a6d45e55a service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:44.550157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1071/2119] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:49.936301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6acbc-1d52-47d4-91bd-184670af3285 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:50.006656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6acbc-1d52-47d4-91bd-184670af3285 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:50.012108 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2b5ee-9815-4b98-beb5-38f67e42b5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:50.012928 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2b5ee-9815-4b98-beb5-38f67e42b5d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:50.013270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2b5ee-9815-4b98-beb5-38f67e42b5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:50.013525 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2b5ee-9815-4b98-beb5-38f67e42b5d7 admin admin] PATH_INFO: `/v3/users/4fb1d748076d44b7a5cfee53ed06c29e` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:50.014711 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2b5ee-9815-4b98-beb5-38f67e42b5d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fb1d748076d44b7a5cfee53ed06c29e)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:50.016778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2b5ee-9815-4b98-beb5-38f67e42b5d7 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:50.073204 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e2c2b5ee-9815-4b98-beb5-38f67e42b5d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fb1d748076d44b7a5cfee53ed06c29e'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:50.074630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e2c2b5ee-9815-4b98-beb5-38f67e42b5d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fb1d748076d44b7a5cfee53ed06c29e'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:50.075285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e2c2b5ee-9815-4b98-beb5-38f67e42b5d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fb1d748076d44b7a5cfee53ed06c29e'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:50.087616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e2c2b5ee-9815-4b98-beb5-38f67e42b5d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fb1d748076d44b7a5cfee53ed06c29e'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:50.104991 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e2c2b5ee-9815-4b98-beb5-38f67e42b5d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fb1d748076d44b7a5cfee53ed06c29e'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:50.121621 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1049/2120] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:49 2022] DELETE /identity/v3/users/4fb1d748076d44b7a5cfee53ed06c29e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:50.126386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024bbdb-6ad8-4f8c-b3f4-22c793856ab3 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:50.127184 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024bbdb-6ad8-4f8c-b3f4-22c793856ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:50.127184 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024bbdb-6ad8-4f8c-b3f4-22c793856ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:50.152346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-1024bbdb-6ad8-4f8c-b3f4-22c793856ab3 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:50.154147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-1024bbdb-6ad8-4f8c-b3f4-22c793856ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:50.224095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1072/2121] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:50.234678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2b5ee-9815-4b98-beb5-38f67e42b5d7 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:50.236175 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e2c2b5ee-9815-4b98-beb5-38f67e42b5d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:50.246955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:50.306788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2b5ee-9815-4b98-beb5-38f67e42b5d7 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:50.310571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769c71f-cac6-47e5-8b76-b7d7bc8d0d86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:50.311207 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769c71f-cac6-47e5-8b76-b7d7bc8d0d86 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:50.311508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769c71f-cac6-47e5-8b76-b7d7bc8d0d86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:50.311779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769c71f-cac6-47e5-8b76-b7d7bc8d0d86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:50.322444 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769c71f-cac6-47e5-8b76-b7d7bc8d0d86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:50.323953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769c71f-cac6-47e5-8b76-b7d7bc8d0d86 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:50.335506 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1050/2122] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:50.690256 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024bbdb-6ad8-4f8c-b3f4-22c793856ab3 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:50.692096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-1024bbdb-6ad8-4f8c-b3f4-22c793856ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:50.704889 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024bbdb-6ad8-4f8c-b3f4-22c793856ab3 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:50.711283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9225db-57a2-4812-a6cb-1bf29c9f98f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:50.711955 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9225db-57a2-4812-a6cb-1bf29c9f98f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:50.712232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9225db-57a2-4812-a6cb-1bf29c9f98f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:50.712500 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9225db-57a2-4812-a6cb-1bf29c9f98f2 admin admin] PATH_INFO: `/v3/projects/47fe5a2568ba483891d6200556f1f4ec` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:50.713730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9225db-57a2-4812-a6cb-1bf29c9f98f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47fe5a2568ba483891d6200556f1f4ec)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:50.715795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9225db-57a2-4812-a6cb-1bf29c9f98f2 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:50.766579 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-cb9225db-57a2-4812-a6cb-1bf29c9f98f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47fe5a2568ba483891d6200556f1f4ec', 'request_id': 'req-cb9225db-57a2-4812-a6cb-1bf29c9f98f2'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:50.772857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-cb9225db-57a2-4812-a6cb-1bf29c9f98f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47fe5a2568ba483891d6200556f1f4ec', 'request_id': 'req-cb9225db-57a2-4812-a6cb-1bf29c9f98f2'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:50.782410 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-cb9225db-57a2-4812-a6cb-1bf29c9f98f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47fe5a2568ba483891d6200556f1f4ec', 'request_id': 'req-cb9225db-57a2-4812-a6cb-1bf29c9f98f2'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:50.796758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-cb9225db-57a2-4812-a6cb-1bf29c9f98f2 admin admin] The token cache is being invalidate because project 47fe5a2568ba483891d6200556f1f4ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:50.797040 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-cb9225db-57a2-4812-a6cb-1bf29c9f98f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:50.797710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-cb9225db-57a2-4812-a6cb-1bf29c9f98f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:50.799808 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1073/2123] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:50 2022] DELETE /identity/v3/projects/47fe5a2568ba483891d6200556f1f4ec => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:50.805061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617e9d8-a2dc-49cb-b9fd-cd97b7c80762 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:50.805348 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617e9d8-a2dc-49cb-b9fd-cd97b7c80762 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:50.805627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617e9d8-a2dc-49cb-b9fd-cd97b7c80762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:50.827602 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-e617e9d8-a2dc-49cb-b9fd-cd97b7c80762 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:50.828851 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e617e9d8-a2dc-49cb-b9fd-cd97b7c80762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:50.946326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1051/2124] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:50.948578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c14b3-0a09-4481-94b4-9c1032e94928 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:50.948899 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c14b3-0a09-4481-94b4-9c1032e94928 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:50.949132 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c14b3-0a09-4481-94b4-9c1032e94928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:50.966097 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:50.974946 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-2c5c14b3-0a09-4481-94b4-9c1032e94928 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:50.977061 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-2c5c14b3-0a09-4481-94b4-9c1032e94928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:51.003582 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1074/2125] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:51.003864 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617e9d8-a2dc-49cb-b9fd-cd97b7c80762 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:51.016459 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617e9d8-a2dc-49cb-b9fd-cd97b7c80762 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:51.022423 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0aca90-0802-4d1e-b49a-07e566ca9bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:51.023220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0aca90-0802-4d1e-b49a-07e566ca9bce admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:51.023566 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0aca90-0802-4d1e-b49a-07e566ca9bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:51.023857 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0aca90-0802-4d1e-b49a-07e566ca9bce admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:51.024506 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0aca90-0802-4d1e-b49a-07e566ca9bce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:51.024811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0aca90-0802-4d1e-b49a-07e566ca9bce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:51.026436 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0aca90-0802-4d1e-b49a-07e566ca9bce admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:51.039030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1052/2126] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:51.043331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c14b3-0a09-4481-94b4-9c1032e94928 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:51.059307 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c14b3-0a09-4481-94b4-9c1032e94928 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:51.065571 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0edb705-5a21-44cd-a38d-7d001e01c875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:51.066629 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edb705-5a21-44cd-a38d-7d001e01c875 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:51.066980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edb705-5a21-44cd-a38d-7d001e01c875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:51.067575 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edb705-5a21-44cd-a38d-7d001e01c875 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:51.068114 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0edb705-5a21-44cd-a38d-7d001e01c875 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:51.070065 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0edb705-5a21-44cd-a38d-7d001e01c875 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:51.103444 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1075/2127] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:51 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:51.106786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0aca90-0802-4d1e-b49a-07e566ca9bce admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:51.121575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0aca90-0802-4d1e-b49a-07e566ca9bce admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:51.187755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21b526-fafa-42de-bba3-69dec7595e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:51.188354 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21b526-fafa-42de-bba3-69dec7595e3f admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:51.188632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21b526-fafa-42de-bba3-69dec7595e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:51.188903 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21b526-fafa-42de-bba3-69dec7595e3f admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:51.189545 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21b526-fafa-42de-bba3-69dec7595e3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:51.191539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21b526-fafa-42de-bba3-69dec7595e3f admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:51.221390 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1053/2128] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:51 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:51.225516 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0edb705-5a21-44cd-a38d-7d001e01c875 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:51.237008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0edb705-5a21-44cd-a38d-7d001e01c875 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:51.245225 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567f940-5bd7-4f97-b882-4c5388160826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:51.245858 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567f940-5bd7-4f97-b882-4c5388160826 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:51.246452 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567f940-5bd7-4f97-b882-4c5388160826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:51.246970 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567f940-5bd7-4f97-b882-4c5388160826 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:51.247723 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567f940-5bd7-4f97-b882-4c5388160826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:51.249673 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567f940-5bd7-4f97-b882-4c5388160826 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:51.265013 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1076/2129] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:51.268883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21b526-fafa-42de-bba3-69dec7595e3f admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:51.281821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21b526-fafa-42de-bba3-69dec7595e3f admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:51.287057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d95b0-ca04-4542-a7bd-fe11ad1dd78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:51.287722 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d95b0-ca04-4542-a7bd-fe11ad1dd78e admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:51.288003 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d95b0-ca04-4542-a7bd-fe11ad1dd78e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:51.288281 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d95b0-ca04-4542-a7bd-fe11ad1dd78e admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:51.289016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d95b0-ca04-4542-a7bd-fe11ad1dd78e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:51.290699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d95b0-ca04-4542-a7bd-fe11ad1dd78e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:51.302795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1054/2130] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:51.306919 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567f940-5bd7-4f97-b882-4c5388160826 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:51.318133 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567f940-5bd7-4f97-b882-4c5388160826 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:51.323165 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e5f4e-1668-4d1b-8d19-4915035fc235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:51.323869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e5f4e-1668-4d1b-8d19-4915035fc235 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:51.324147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e5f4e-1668-4d1b-8d19-4915035fc235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:51.324417 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e5f4e-1668-4d1b-8d19-4915035fc235 admin admin] PATH_INFO: `/v3/projects/0a2ef3b6384a4194a26ba51078c425af/users/9f4d65ffaab942f29d449b64805c765a/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:51.342344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e5f4e-1668-4d1b-8d19-4915035fc235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2ef3b6384a4194a26ba51078c425af, user_id=9f4d65ffaab942f29d449b64805c765a, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:51.345223 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e5f4e-1668-4d1b-8d19-4915035fc235 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:51.367137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1077/2131] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:51 2022] PUT /identity/v3/projects/0a2ef3b6384a4194a26ba51078c425af/users/9f4d65ffaab942f29d449b64805c765a/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:51.372342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181ee0e-360e-482b-922e-0aad9338d8f0 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:51.372658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181ee0e-360e-482b-922e-0aad9338d8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:51.372903 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181ee0e-360e-482b-922e-0aad9338d8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:51.392778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-5181ee0e-360e-482b-922e-0aad9338d8f0 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:51.394031 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-5181ee0e-360e-482b-922e-0aad9338d8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:51.405198 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:51.456680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1055/2132] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:51.467344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e5f4e-1668-4d1b-8d19-4915035fc235 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:51.469366 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-245e5f4e-1668-4d1b-8d19-4915035fc235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:51.540793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e5f4e-1668-4d1b-8d19-4915035fc235 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:51.545949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaef75d-d6f2-464c-ba1d-f018d15b019c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:51.546706 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaef75d-d6f2-464c-ba1d-f018d15b019c service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:51.546983 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaef75d-d6f2-464c-ba1d-f018d15b019c service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:51.547255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaef75d-d6f2-464c-ba1d-f018d15b019c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:51.557592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaef75d-d6f2-464c-ba1d-f018d15b019c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:51.559040 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaef75d-d6f2-464c-ba1d-f018d15b019c service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:51.570401 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1078/2133] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:52.035004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181ee0e-360e-482b-922e-0aad9338d8f0 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:52.037630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-5181ee0e-360e-482b-922e-0aad9338d8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:52.053477 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181ee0e-360e-482b-922e-0aad9338d8f0 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:52.059743 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11763c54-fe8a-4abd-9d0c-da1bc032db7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:52.060788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11763c54-fe8a-4abd-9d0c-da1bc032db7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:52.061250 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11763c54-fe8a-4abd-9d0c-da1bc032db7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:52.061937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11763c54-fe8a-4abd-9d0c-da1bc032db7c admin admin] PATH_INFO: `/v3/users/301a1e1105ea4c1b9b32336793df73a1` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:52.064441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11763c54-fe8a-4abd-9d0c-da1bc032db7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=301a1e1105ea4c1b9b32336793df73a1)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:52.068357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11763c54-fe8a-4abd-9d0c-da1bc032db7c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:52.094715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaef75d-d6f2-464c-ba1d-f018d15b019c service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:52.096677 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-cdaef75d-d6f2-464c-ba1d-f018d15b019c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:52.120559 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaef75d-d6f2-464c-ba1d-f018d15b019c service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:52.138376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266254ee-c1f7-4d81-901e-34d788f2decc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:52.139888 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266254ee-c1f7-4d81-901e-34d788f2decc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:52.141498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266254ee-c1f7-4d81-901e-34d788f2decc admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:52.141766 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266254ee-c1f7-4d81-901e-34d788f2decc admin admin] PATH_INFO: `/v3/users/b740bcefdbd243fcbe1e49442275d347` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:52.146031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266254ee-c1f7-4d81-901e-34d788f2decc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b740bcefdbd243fcbe1e49442275d347)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:52.154586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266254ee-c1f7-4d81-901e-34d788f2decc admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:52.181793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-11763c54-fe8a-4abd-9d0c-da1bc032db7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '301a1e1105ea4c1b9b32336793df73a1'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:52.182559 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-11763c54-fe8a-4abd-9d0c-da1bc032db7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '301a1e1105ea4c1b9b32336793df73a1'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:52.183167 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-11763c54-fe8a-4abd-9d0c-da1bc032db7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '301a1e1105ea4c1b9b32336793df73a1'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:52.193131 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-11763c54-fe8a-4abd-9d0c-da1bc032db7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '301a1e1105ea4c1b9b32336793df73a1'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:52.209458 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-11763c54-fe8a-4abd-9d0c-da1bc032db7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '301a1e1105ea4c1b9b32336793df73a1'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:52.237429 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1056/2134] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:52 2022] DELETE /identity/v3/users/301a1e1105ea4c1b9b32336793df73a1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:52.243249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6f6fc-5dee-420a-b674-ef9588e897d0 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:52.245555 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6f6fc-5dee-420a-b674-ef9588e897d0 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:52.245852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6f6fc-5dee-420a-b674-ef9588e897d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:52.275890 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-266254ee-c1f7-4d81-901e-34d788f2decc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b740bcefdbd243fcbe1e49442275d347'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:52.277531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-266254ee-c1f7-4d81-901e-34d788f2decc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b740bcefdbd243fcbe1e49442275d347'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:52.278353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-266254ee-c1f7-4d81-901e-34d788f2decc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b740bcefdbd243fcbe1e49442275d347'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:52.292813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-e0a6f6fc-5dee-420a-b674-ef9588e897d0 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:52.295496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e0a6f6fc-5dee-420a-b674-ef9588e897d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:52.297700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-266254ee-c1f7-4d81-901e-34d788f2decc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b740bcefdbd243fcbe1e49442275d347'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:52.326110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-266254ee-c1f7-4d81-901e-34d788f2decc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b740bcefdbd243fcbe1e49442275d347'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:52.355929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1079/2135] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:52 2022] DELETE /identity/v3/users/b740bcefdbd243fcbe1e49442275d347 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:52.361161 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086af58-e332-435a-aea3-ea37137d9123 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:52.361638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086af58-e332-435a-aea3-ea37137d9123 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:52.362301 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086af58-e332-435a-aea3-ea37137d9123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:52.392920 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-b086af58-e332-435a-aea3-ea37137d9123 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:52.394797 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b086af58-e332-435a-aea3-ea37137d9123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:52.416805 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:52.439807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1057/2136] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:52.462950 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6f6fc-5dee-420a-b674-ef9588e897d0 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:52.465514 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e0a6f6fc-5dee-420a-b674-ef9588e897d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:52.523126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1080/2137] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:52.523126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8633374-58dd-4d14-b56b-b77bd0744228 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:52.523126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8633374-58dd-4d14-b56b-b77bd0744228 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:52.523126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8633374-58dd-4d14-b56b-b77bd0744228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:52.540491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:52.561033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-c8633374-58dd-4d14-b56b-b77bd0744228 None None] MFA Rules not processed for user `9f4d65ffaab942f29d449b64805c765a`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:52.563042 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c8633374-58dd-4d14-b56b-b77bd0744228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:52.586593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6f6fc-5dee-420a-b674-ef9588e897d0 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:52.652958 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1081/2138] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:52.654545 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8633374-58dd-4d14-b56b-b77bd0744228 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:52.656401 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c8633374-58dd-4d14-b56b-b77bd0744228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:52.691387 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001eaa6b-b4ed-40c5-bc7b-bfc8dc0615b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:52.693669 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001eaa6b-b4ed-40c5-bc7b-bfc8dc0615b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:52.693947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001eaa6b-b4ed-40c5-bc7b-bfc8dc0615b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:52.694231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001eaa6b-b4ed-40c5-bc7b-bfc8dc0615b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:52.705163 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001eaa6b-b4ed-40c5-bc7b-bfc8dc0615b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:52.707379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001eaa6b-b4ed-40c5-bc7b-bfc8dc0615b4 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:52.719595 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1058/2139] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:52.720385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001eaa6b-b4ed-40c5-bc7b-bfc8dc0615b4 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:52.721721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-001eaa6b-b4ed-40c5-bc7b-bfc8dc0615b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:52.753137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8633374-58dd-4d14-b56b-b77bd0744228 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:52.759722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b19c8-4fa1-4c96-aba6-fe078fa038e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:52.760355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b19c8-4fa1-4c96-aba6-fe078fa038e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:52.760687 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b19c8-4fa1-4c96-aba6-fe078fa038e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:52.760965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b19c8-4fa1-4c96-aba6-fe078fa038e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:52.785167 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b19c8-4fa1-4c96-aba6-fe078fa038e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:52.789429 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b19c8-4fa1-4c96-aba6-fe078fa038e8 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:52.806816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1082/2140] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:52.846670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001eaa6b-b4ed-40c5-bc7b-bfc8dc0615b4 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:52.853355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d1aa2-dc18-4072-a567-a47a7ca4a15c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:52.854044 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d1aa2-dc18-4072-a567-a47a7ca4a15c service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:52.854447 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d1aa2-dc18-4072-a567-a47a7ca4a15c service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:52.854780 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d1aa2-dc18-4072-a567-a47a7ca4a15c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:52.885161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d1aa2-dc18-4072-a567-a47a7ca4a15c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:52.887100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d1aa2-dc18-4072-a567-a47a7ca4a15c service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:52.900685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1059/2141] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:53.217739 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b19c8-4fa1-4c96-aba6-fe078fa038e8 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:53.219524 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b85b19c8-4fa1-4c96-aba6-fe078fa038e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:53.244309 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b19c8-4fa1-4c96-aba6-fe078fa038e8 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:53.265781 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a3e8f-6ebb-444d-9dc9-947099fe4655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:53.267943 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a3e8f-6ebb-444d-9dc9-947099fe4655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:53.268998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a3e8f-6ebb-444d-9dc9-947099fe4655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:53.270308 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a3e8f-6ebb-444d-9dc9-947099fe4655 admin admin] PATH_INFO: `/v3/projects/857e3448f8ec445bbba6589f2acde255` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:53.273896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a3e8f-6ebb-444d-9dc9-947099fe4655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=857e3448f8ec445bbba6589f2acde255)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:53.276500 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a3e8f-6ebb-444d-9dc9-947099fe4655 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:53.377148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:53.379638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-df5a3e8f-6ebb-444d-9dc9-947099fe4655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '857e3448f8ec445bbba6589f2acde255', 'request_id': 'req-df5a3e8f-6ebb-444d-9dc9-947099fe4655'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:53.387635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-df5a3e8f-6ebb-444d-9dc9-947099fe4655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '857e3448f8ec445bbba6589f2acde255', 'request_id': 'req-df5a3e8f-6ebb-444d-9dc9-947099fe4655'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:53.399609 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-df5a3e8f-6ebb-444d-9dc9-947099fe4655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '857e3448f8ec445bbba6589f2acde255', 'request_id': 'req-df5a3e8f-6ebb-444d-9dc9-947099fe4655'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:53.416379 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-df5a3e8f-6ebb-444d-9dc9-947099fe4655 admin admin] The token cache is being invalidate because project 857e3448f8ec445bbba6589f2acde255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:53.416786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-df5a3e8f-6ebb-444d-9dc9-947099fe4655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:53.418014 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-df5a3e8f-6ebb-444d-9dc9-947099fe4655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:53.423628 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1083/2142] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:53 2022] DELETE /identity/v3/projects/857e3448f8ec445bbba6589f2acde255 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:53.423883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d1aa2-dc18-4072-a567-a47a7ca4a15c service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:53.450438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d1aa2-dc18-4072-a567-a47a7ca4a15c service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:53.527554 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304f94a-4777-4921-9416-d24cbcf5a3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:53.528235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304f94a-4777-4921-9416-d24cbcf5a3ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:53.528521 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304f94a-4777-4921-9416-d24cbcf5a3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:53.528794 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304f94a-4777-4921-9416-d24cbcf5a3ad admin admin] PATH_INFO: `/v3/users/bf031e28f7d84f2b80f8f4940ccf1889` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:53.529953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304f94a-4777-4921-9416-d24cbcf5a3ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf031e28f7d84f2b80f8f4940ccf1889)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:53.532008 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304f94a-4777-4921-9416-d24cbcf5a3ad admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:53.577697 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:53.594297 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-5304f94a-4777-4921-9416-d24cbcf5a3ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf031e28f7d84f2b80f8f4940ccf1889'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:53.595285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-5304f94a-4777-4921-9416-d24cbcf5a3ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf031e28f7d84f2b80f8f4940ccf1889'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:53.595850 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-5304f94a-4777-4921-9416-d24cbcf5a3ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf031e28f7d84f2b80f8f4940ccf1889'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:53.604554 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-5304f94a-4777-4921-9416-d24cbcf5a3ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf031e28f7d84f2b80f8f4940ccf1889'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:53.616947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-5304f94a-4777-4921-9416-d24cbcf5a3ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf031e28f7d84f2b80f8f4940ccf1889'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:53.632693 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1060/2143] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:53 2022] DELETE /identity/v3/users/bf031e28f7d84f2b80f8f4940ccf1889 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:53.744082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a3e8f-6ebb-444d-9dc9-947099fe4655 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:53.745803 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-df5a3e8f-6ebb-444d-9dc9-947099fe4655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:53.773919 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a3e8f-6ebb-444d-9dc9-947099fe4655 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:53.902592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb715db0-1f8e-4fd2-8f36-cfbce9a674a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:53.904358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb715db0-1f8e-4fd2-8f36-cfbce9a674a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:53.904786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb715db0-1f8e-4fd2-8f36-cfbce9a674a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:53.905201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb715db0-1f8e-4fd2-8f36-cfbce9a674a2 admin admin] PATH_INFO: `/v3/projects/20d1d40307a2438e95cac3a2b118a698` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:53.906595 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb715db0-1f8e-4fd2-8f36-cfbce9a674a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20d1d40307a2438e95cac3a2b118a698)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:53.910743 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb715db0-1f8e-4fd2-8f36-cfbce9a674a2 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:53.989444 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fb715db0-1f8e-4fd2-8f36-cfbce9a674a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20d1d40307a2438e95cac3a2b118a698', 'request_id': 'req-fb715db0-1f8e-4fd2-8f36-cfbce9a674a2'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:53.993880 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304f94a-4777-4921-9416-d24cbcf5a3ad admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:53.995448 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-5304f94a-4777-4921-9416-d24cbcf5a3ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:54.001425 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fb715db0-1f8e-4fd2-8f36-cfbce9a674a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20d1d40307a2438e95cac3a2b118a698', 'request_id': 'req-fb715db0-1f8e-4fd2-8f36-cfbce9a674a2'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:54.015946 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304f94a-4777-4921-9416-d24cbcf5a3ad admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:54.016655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fb715db0-1f8e-4fd2-8f36-cfbce9a674a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20d1d40307a2438e95cac3a2b118a698', 'request_id': 'req-fb715db0-1f8e-4fd2-8f36-cfbce9a674a2'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:54.061198 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fb715db0-1f8e-4fd2-8f36-cfbce9a674a2 admin admin] The token cache is being invalidate because project 20d1d40307a2438e95cac3a2b118a698 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:54.061493 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fb715db0-1f8e-4fd2-8f36-cfbce9a674a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:54.062277 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-fb715db0-1f8e-4fd2-8f36-cfbce9a674a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:54.069637 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1084/2144] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:53 2022] DELETE /identity/v3/projects/20d1d40307a2438e95cac3a2b118a698 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:54.072011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d66cd-e001-48c5-aa83-5d14bcd69217 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:54.072884 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d66cd-e001-48c5-aa83-5d14bcd69217 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:54.073153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d66cd-e001-48c5-aa83-5d14bcd69217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:54.145363 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-227d66cd-e001-48c5-aa83-5d14bcd69217 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:54.147143 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-227d66cd-e001-48c5-aa83-5d14bcd69217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:54.182064 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f93e8-9ee6-4a87-b2bf-ff87e450c7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:54.183284 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f93e8-9ee6-4a87-b2bf-ff87e450c7e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:54.183699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f93e8-9ee6-4a87-b2bf-ff87e450c7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:54.184072 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f93e8-9ee6-4a87-b2bf-ff87e450c7e6 admin admin] PATH_INFO: `/v3/projects/4cbfac92f8ef46af83fe3a96434870cc` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:54.185431 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f93e8-9ee6-4a87-b2bf-ff87e450c7e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cbfac92f8ef46af83fe3a96434870cc)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:54.187644 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f93e8-9ee6-4a87-b2bf-ff87e450c7e6 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:54.271563 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1085/2145] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:54.285857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf418a2-c141-4da3-bb29-6d47c7826135 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:54.286945 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf418a2-c141-4da3-bb29-6d47c7826135 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:54.286945 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf418a2-c141-4da3-bb29-6d47c7826135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:54.291580 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-ab2f93e8-9ee6-4a87-b2bf-ff87e450c7e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cbfac92f8ef46af83fe3a96434870cc', 'request_id': 'req-ab2f93e8-9ee6-4a87-b2bf-ff87e450c7e6'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:54.305077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-ab2f93e8-9ee6-4a87-b2bf-ff87e450c7e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cbfac92f8ef46af83fe3a96434870cc', 'request_id': 'req-ab2f93e8-9ee6-4a87-b2bf-ff87e450c7e6'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:54.312070 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:54.325049 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-ab2f93e8-9ee6-4a87-b2bf-ff87e450c7e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cbfac92f8ef46af83fe3a96434870cc', 'request_id': 'req-ab2f93e8-9ee6-4a87-b2bf-ff87e450c7e6'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:54.327892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-edf418a2-c141-4da3-bb29-6d47c7826135 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:54.330031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-edf418a2-c141-4da3-bb29-6d47c7826135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:54.354588 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-ab2f93e8-9ee6-4a87-b2bf-ff87e450c7e6 admin admin] The token cache is being invalidate because project 4cbfac92f8ef46af83fe3a96434870cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:54.355018 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-ab2f93e8-9ee6-4a87-b2bf-ff87e450c7e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:54.355985 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-ab2f93e8-9ee6-4a87-b2bf-ff87e450c7e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:54.357995 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1061/2146] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:53 2022] DELETE /identity/v3/projects/4cbfac92f8ef46af83fe3a96434870cc => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:54.359252 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f93e8-9ee6-4a87-b2bf-ff87e450c7e6 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:54.364948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ab2f93e8-9ee6-4a87-b2bf-ff87e450c7e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:54.438905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1086/2147] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:54.443685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0eef05-87d0-4306-9b65-efc95bd5b041 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:54.443972 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0eef05-87d0-4306-9b65-efc95bd5b041 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:54.444856 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0eef05-87d0-4306-9b65-efc95bd5b041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:54.454580 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:54.487200 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f93e8-9ee6-4a87-b2bf-ff87e450c7e6 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:54.492481 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-ce0eef05-87d0-4306-9b65-efc95bd5b041 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:54.493226 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90717c27-c9c7-4c6f-97ca-7ecf9340201e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:54.493699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ce0eef05-87d0-4306-9b65-efc95bd5b041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:54.494070 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90717c27-c9c7-4c6f-97ca-7ecf9340201e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:54.495003 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90717c27-c9c7-4c6f-97ca-7ecf9340201e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:54.495375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90717c27-c9c7-4c6f-97ca-7ecf9340201e admin admin] PATH_INFO: `/v3/users/599ff9e636ee4a8d976208ca0854bda7` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:54.496702 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90717c27-c9c7-4c6f-97ca-7ecf9340201e admin admin] RBAC: Authorizing `identity:delete_user(user_id=599ff9e636ee4a8d976208ca0854bda7)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:54.498780 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90717c27-c9c7-4c6f-97ca-7ecf9340201e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:54.512190 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1087/2148] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:54.513276 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0eef05-87d0-4306-9b65-efc95bd5b041 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:54.529404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0eef05-87d0-4306-9b65-efc95bd5b041 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:54.536855 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce58d1-aa7d-4e86-a673-e22b3735cbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:54.538864 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce58d1-aa7d-4e86-a673-e22b3735cbfb admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:54.539295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce58d1-aa7d-4e86-a673-e22b3735cbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:54.539731 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce58d1-aa7d-4e86-a673-e22b3735cbfb admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:54.541313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce58d1-aa7d-4e86-a673-e22b3735cbfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:54.542157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce58d1-aa7d-4e86-a673-e22b3735cbfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:54.544327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce58d1-aa7d-4e86-a673-e22b3735cbfb admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:54.572180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1088/2149] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:54.573687 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8a91c-6638-4b9f-93c4-32c6919ffa93 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:54.574074 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8a91c-6638-4b9f-93c4-32c6919ffa93 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:54.574442 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8a91c-6638-4b9f-93c4-32c6919ffa93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:54.575854 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-90717c27-c9c7-4c6f-97ca-7ecf9340201e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '599ff9e636ee4a8d976208ca0854bda7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:54.576876 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-90717c27-c9c7-4c6f-97ca-7ecf9340201e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '599ff9e636ee4a8d976208ca0854bda7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:54.579024 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-90717c27-c9c7-4c6f-97ca-7ecf9340201e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '599ff9e636ee4a8d976208ca0854bda7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:54.593539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-90717c27-c9c7-4c6f-97ca-7ecf9340201e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '599ff9e636ee4a8d976208ca0854bda7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:54.607784 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-6cb8a91c-6638-4b9f-93c4-32c6919ffa93 None None] MFA Rules not processed for user `9f4d65ffaab942f29d449b64805c765a`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:54.609660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6cb8a91c-6638-4b9f-93c4-32c6919ffa93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:54.611891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-90717c27-c9c7-4c6f-97ca-7ecf9340201e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '599ff9e636ee4a8d976208ca0854bda7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:54.626515 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1062/2150] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:54 2022] DELETE /identity/v3/users/599ff9e636ee4a8d976208ca0854bda7 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:54.627890 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f1228-67fe-4190-9936-4d702b35cb17 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:54.628175 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f1228-67fe-4190-9936-4d702b35cb17 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:54.628446 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f1228-67fe-4190-9936-4d702b35cb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:54.649558 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-c17f1228-67fe-4190-9936-4d702b35cb17 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:54.654640 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c17f1228-67fe-4190-9936-4d702b35cb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:54.697462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1089/2151] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:54.702610 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb8a91c-6638-4b9f-93c4-32c6919ffa93 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:54.702610 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6cb8a91c-6638-4b9f-93c4-32c6919ffa93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:54.719887 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb8a91c-6638-4b9f-93c4-32c6919ffa93 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:54.754132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1063/2152] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:54.760409 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d87ed7-9a08-4a36-95e4-4f046301f5e7 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:54.760409 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d87ed7-9a08-4a36-95e4-4f046301f5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:54.760681 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d87ed7-9a08-4a36-95e4-4f046301f5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:54.789020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fb45c-39e4-433c-9f00-7dce2c6fca5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:54.789633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fb45c-39e4-433c-9f00-7dce2c6fca5c admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:54.790116 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fb45c-39e4-433c-9f00-7dce2c6fca5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:54.790399 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fb45c-39e4-433c-9f00-7dce2c6fca5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:54.791215 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99fb45c-39e4-433c-9f00-7dce2c6fca5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:54.791579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-85d87ed7-9a08-4a36-95e4-4f046301f5e7 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:54.792822 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-85d87ed7-9a08-4a36-95e4-4f046301f5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:54.793595 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99fb45c-39e4-433c-9f00-7dce2c6fca5c admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:54.802021 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1064/2153] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:54.802914 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d87ed7-9a08-4a36-95e4-4f046301f5e7 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:54.814281 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d87ed7-9a08-4a36-95e4-4f046301f5e7 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:54.820800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8eaba7-902b-4383-8549-0a9f24136421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:54.821449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8eaba7-902b-4383-8549-0a9f24136421 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:54.821804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8eaba7-902b-4383-8549-0a9f24136421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:54.822238 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8eaba7-902b-4383-8549-0a9f24136421 admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:54.822904 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8eaba7-902b-4383-8549-0a9f24136421 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:54.823209 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8eaba7-902b-4383-8549-0a9f24136421 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:54.824915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8eaba7-902b-4383-8549-0a9f24136421 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:54.841424 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1090/2154] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:54 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:54.842642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fb45c-39e4-433c-9f00-7dce2c6fca5c admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:54.843598 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1065/2155] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:54.845392 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a99fb45c-39e4-433c-9f00-7dce2c6fca5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:54.846190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8eaba7-902b-4383-8549-0a9f24136421 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:54.870741 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:54.875725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8eaba7-902b-4383-8549-0a9f24136421 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:54.932787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fb45c-39e4-433c-9f00-7dce2c6fca5c admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:54.941613 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856378fa-f647-4197-a5cf-e18d6a55b0c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:54.943044 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856378fa-f647-4197-a5cf-e18d6a55b0c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:54.943513 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856378fa-f647-4197-a5cf-e18d6a55b0c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:54.943938 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856378fa-f647-4197-a5cf-e18d6a55b0c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:54.944781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:54.971180 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543f8d5-7502-4201-995c-0e446105904b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:54.971833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543f8d5-7502-4201-995c-0e446105904b admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:54.972641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543f8d5-7502-4201-995c-0e446105904b admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:54.973026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543f8d5-7502-4201-995c-0e446105904b admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:54.974367 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543f8d5-7502-4201-995c-0e446105904b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:54.976683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543f8d5-7502-4201-995c-0e446105904b admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:55.015333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1066/2156] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:54 2022] POST /identity/v3/users => generated 283 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:55.016462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543f8d5-7502-4201-995c-0e446105904b admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:55.023349 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856378fa-f647-4197-a5cf-e18d6a55b0c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:55.024835 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856378fa-f647-4197-a5cf-e18d6a55b0c4 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:55.028927 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543f8d5-7502-4201-995c-0e446105904b admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:55.034616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10e698-f2ff-415a-8e47-a636625c76f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:55.034892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1091/2157] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:55.035638 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10e698-f2ff-415a-8e47-a636625c76f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:55.038340 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856378fa-f647-4197-a5cf-e18d6a55b0c4 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:55.038467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10e698-f2ff-415a-8e47-a636625c76f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:55.038467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10e698-f2ff-415a-8e47-a636625c76f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:55.038467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb10e698-f2ff-415a-8e47-a636625c76f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:55.038659 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-856378fa-f647-4197-a5cf-e18d6a55b0c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:55.039009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb10e698-f2ff-415a-8e47-a636625c76f5 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:55.050415 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856378fa-f647-4197-a5cf-e18d6a55b0c4 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:55.055282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f422f3-c9f7-4f8c-908c-b14b11e76356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:55.055956 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f422f3-c9f7-4f8c-908c-b14b11e76356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:55.056236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f422f3-c9f7-4f8c-908c-b14b11e76356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:55.056510 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f422f3-c9f7-4f8c-908c-b14b11e76356 admin admin] PATH_INFO: `/v3/users/6cbcc9668f454c0e8c27495a1827d703` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:55.057844 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f422f3-c9f7-4f8c-908c-b14b11e76356 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cbcc9668f454c0e8c27495a1827d703)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:55.060670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f422f3-c9f7-4f8c-908c-b14b11e76356 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:55.077689 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1067/2158] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:55 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:55.077689 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10e698-f2ff-415a-8e47-a636625c76f5 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:55.120962 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c3f422f3-c9f7-4f8c-908c-b14b11e76356 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cbcc9668f454c0e8c27495a1827d703'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:55.122687 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c3f422f3-c9f7-4f8c-908c-b14b11e76356 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cbcc9668f454c0e8c27495a1827d703'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:55.123622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c3f422f3-c9f7-4f8c-908c-b14b11e76356 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cbcc9668f454c0e8c27495a1827d703'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:55.135179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c3f422f3-c9f7-4f8c-908c-b14b11e76356 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cbcc9668f454c0e8c27495a1827d703'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:55.149394 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c3f422f3-c9f7-4f8c-908c-b14b11e76356 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cbcc9668f454c0e8c27495a1827d703'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:55.159616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10e698-f2ff-415a-8e47-a636625c76f5 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:55.166229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb069a-bcbe-40a8-890f-383de47293c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:55.166851 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb069a-bcbe-40a8-890f-383de47293c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:55.167287 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb069a-bcbe-40a8-890f-383de47293c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:55.167585 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb069a-bcbe-40a8-890f-383de47293c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:55.168413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb069a-bcbe-40a8-890f-383de47293c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:55.169063 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1092/2159] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:13:55 2022] DELETE /identity/v3/users/6cbcc9668f454c0e8c27495a1827d703 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:55.170420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb069a-bcbe-40a8-890f-383de47293c5 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:55.173045 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f422f3-c9f7-4f8c-908c-b14b11e76356 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:55.175218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c3f422f3-c9f7-4f8c-908c-b14b11e76356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:55.194629 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1068/2160] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:55.199137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb069a-bcbe-40a8-890f-383de47293c5 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:55.202209 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-aafb069a-bcbe-40a8-890f-383de47293c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:55.224036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb069a-bcbe-40a8-890f-383de47293c5 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:55.229213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:55.286757 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f422f3-c9f7-4f8c-908c-b14b11e76356 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:55.294623 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f6551-4f86-4869-bcb1-e4cbeb28725d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:55.296263 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f6551-4f86-4869-bcb1-e4cbeb28725d admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:55.297449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f6551-4f86-4869-bcb1-e4cbeb28725d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:55.297929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f6551-4f86-4869-bcb1-e4cbeb28725d admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:55.299390 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f6551-4f86-4869-bcb1-e4cbeb28725d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:55.303085 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f6551-4f86-4869-bcb1-e4cbeb28725d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:55.303671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50a30e-62b6-4553-9a97-e269edc3b6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:55.304304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50a30e-62b6-4553-9a97-e269edc3b6bd admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:55.304584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50a30e-62b6-4553-9a97-e269edc3b6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:55.304938 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50a30e-62b6-4553-9a97-e269edc3b6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:55.305884 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50a30e-62b6-4553-9a97-e269edc3b6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:55.307901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50a30e-62b6-4553-9a97-e269edc3b6bd admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:55.320063 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1069/2161] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:55.323983 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50a30e-62b6-4553-9a97-e269edc3b6bd admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:55.337511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50a30e-62b6-4553-9a97-e269edc3b6bd admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:55.343275 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679b291-126e-4daf-bbd5-8988953c237b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:55.344393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679b291-126e-4daf-bbd5-8988953c237b admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:55.344777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679b291-126e-4daf-bbd5-8988953c237b admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:55.345353 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679b291-126e-4daf-bbd5-8988953c237b admin admin] PATH_INFO: `/v3/projects/cc42282f6bce4842bc40e07475315514/users/df1ac13407df46088eaf031e7c12ec3a/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:55.351187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1093/2162] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:55 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:55.352399 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f6551-4f86-4869-bcb1-e4cbeb28725d admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:55.365664 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679b291-126e-4daf-bbd5-8988953c237b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc42282f6bce4842bc40e07475315514, user_id=df1ac13407df46088eaf031e7c12ec3a, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:55.369010 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679b291-126e-4daf-bbd5-8988953c237b admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:55.370691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f6551-4f86-4869-bcb1-e4cbeb28725d admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:55.383295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2b590-a9d6-45a6-8845-4ae70dfd143b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:55.384148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2b590-a9d6-45a6-8845-4ae70dfd143b admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:55.384428 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2b590-a9d6-45a6-8845-4ae70dfd143b admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:55.384701 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2b590-a9d6-45a6-8845-4ae70dfd143b admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:55.385404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2b590-a9d6-45a6-8845-4ae70dfd143b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:55.387114 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2b590-a9d6-45a6-8845-4ae70dfd143b admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:55.396732 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1070/2163] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:55 2022] PUT /identity/v3/projects/cc42282f6bce4842bc40e07475315514/users/df1ac13407df46088eaf031e7c12ec3a/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:55.411631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8873923-db1a-4cea-89b7-cdf5b6db38fa None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:55.411961 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8873923-db1a-4cea-89b7-cdf5b6db38fa None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:55.412225 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8873923-db1a-4cea-89b7-cdf5b6db38fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:55.420937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1094/2164] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:55.421954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2b590-a9d6-45a6-8845-4ae70dfd143b admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:55.435447 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2b590-a9d6-45a6-8845-4ae70dfd143b admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:55.500971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-c8873923-db1a-4cea-89b7-cdf5b6db38fa None None] MFA Rules not processed for user `df1ac13407df46088eaf031e7c12ec3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:55.502651 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c8873923-db1a-4cea-89b7-cdf5b6db38fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:55.507216 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:55.561615 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699950a-ddec-4033-8a68-15cc78d6b7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:55.562546 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699950a-ddec-4033-8a68-15cc78d6b7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:55.563023 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699950a-ddec-4033-8a68-15cc78d6b7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:55.563565 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699950a-ddec-4033-8a68-15cc78d6b7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:55.564787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0699950a-ddec-4033-8a68-15cc78d6b7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:55.566815 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0699950a-ddec-4033-8a68-15cc78d6b7b9 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:55.579039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1095/2165] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:55.580132 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699950a-ddec-4033-8a68-15cc78d6b7b9 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:55.582826 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-0699950a-ddec-4033-8a68-15cc78d6b7b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:55.597324 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699950a-ddec-4033-8a68-15cc78d6b7b9 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:55.599238 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1071/2166] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:55.600749 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8873923-db1a-4cea-89b7-cdf5b6db38fa None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:55.612134 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55769a-5a19-43eb-a351-73511870cf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:55.612908 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55769a-5a19-43eb-a351-73511870cf5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:55.613190 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55769a-5a19-43eb-a351-73511870cf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:55.613562 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55769a-5a19-43eb-a351-73511870cf5e admin admin] PATH_INFO: `/v3/projects/24307daab4d24f979c30e081163c3bfe` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:55.614987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55769a-5a19-43eb-a351-73511870cf5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=24307daab4d24f979c30e081163c3bfe)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:55.616971 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55769a-5a19-43eb-a351-73511870cf5e admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:55.623579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8873923-db1a-4cea-89b7-cdf5b6db38fa None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:55.634069 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8093db-56b6-48d6-ae89-f80bee0b17db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:55.635173 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8093db-56b6-48d6-ae89-f80bee0b17db admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:55.635940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8093db-56b6-48d6-ae89-f80bee0b17db admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:55.636272 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8093db-56b6-48d6-ae89-f80bee0b17db admin admin] PATH_INFO: `/v3/projects/432b19722b6a4884a847872a3e181b49/users/2573fe4a125049979ce42bfff8672a13/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:55.672277 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8093db-56b6-48d6-ae89-f80bee0b17db admin admin] RBAC: Authorizing `identity:create_grant(project_id=432b19722b6a4884a847872a3e181b49, user_id=2573fe4a125049979ce42bfff8672a13, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:55.677238 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8093db-56b6-48d6-ae89-f80bee0b17db admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:55.703117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1072/2167] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:55 2022] PUT /identity/v3/projects/432b19722b6a4884a847872a3e181b49/users/2573fe4a125049979ce42bfff8672a13/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:55.704218 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8093db-56b6-48d6-ae89-f80bee0b17db admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:55.710007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-bd8093db-56b6-48d6-ae89-f80bee0b17db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:55.710853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1a55769a-5a19-43eb-a351-73511870cf5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24307daab4d24f979c30e081163c3bfe', 'request_id': 'req-1a55769a-5a19-43eb-a351-73511870cf5e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:55.719160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1a55769a-5a19-43eb-a351-73511870cf5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24307daab4d24f979c30e081163c3bfe', 'request_id': 'req-1a55769a-5a19-43eb-a351-73511870cf5e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:55.730661 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1a55769a-5a19-43eb-a351-73511870cf5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24307daab4d24f979c30e081163c3bfe', 'request_id': 'req-1a55769a-5a19-43eb-a351-73511870cf5e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:55.745686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1a55769a-5a19-43eb-a351-73511870cf5e admin admin] The token cache is being invalidate because project 24307daab4d24f979c30e081163c3bfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:55.745977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1a55769a-5a19-43eb-a351-73511870cf5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:55.746653 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1a55769a-5a19-43eb-a351-73511870cf5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:55.748209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1096/2168] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:55 2022] DELETE /identity/v3/projects/24307daab4d24f979c30e081163c3bfe => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:55.749050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55769a-5a19-43eb-a351-73511870cf5e admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:55.750660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-1a55769a-5a19-43eb-a351-73511870cf5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:55.780417 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8093db-56b6-48d6-ae89-f80bee0b17db admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:55.785856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927e69a-9958-4208-a7fd-423527e56611 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:55.786567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927e69a-9958-4208-a7fd-423527e56611 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:55.786993 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927e69a-9958-4208-a7fd-423527e56611 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:55.787275 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927e69a-9958-4208-a7fd-423527e56611 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:55.789271 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e927e69a-9958-4208-a7fd-423527e56611 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:55.799641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:55.831813 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55769a-5a19-43eb-a351-73511870cf5e admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:55.854946 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e927e69a-9958-4208-a7fd-423527e56611 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:55.856458 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e927e69a-9958-4208-a7fd-423527e56611 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:55.865792 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1073/2169] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:55.867108 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57600d86-9732-429f-b23d-b531e6cd455e None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:55.867353 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57600d86-9732-429f-b23d-b531e6cd455e None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:55.867667 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57600d86-9732-429f-b23d-b531e6cd455e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:55.899845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-57600d86-9732-429f-b23d-b531e6cd455e None None] MFA Rules not processed for user `2573fe4a125049979ce42bfff8672a13`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:55.900139 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b31eb-beec-4b9d-850d-08e08cafc348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:55.900850 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b31eb-beec-4b9d-850d-08e08cafc348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:55.901246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-57600d86-9732-429f-b23d-b531e6cd455e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:55.901356 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b31eb-beec-4b9d-850d-08e08cafc348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:55.901655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b31eb-beec-4b9d-850d-08e08cafc348 admin admin] PATH_INFO: `/v3/projects/59540531d83242cb80c880f2341cfbbc` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:55.902808 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b31eb-beec-4b9d-850d-08e08cafc348 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59540531d83242cb80c880f2341cfbbc)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:55.904788 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b31eb-beec-4b9d-850d-08e08cafc348 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:55.947045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:55.985958 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e45b31eb-beec-4b9d-850d-08e08cafc348 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59540531d83242cb80c880f2341cfbbc', 'request_id': 'req-e45b31eb-beec-4b9d-850d-08e08cafc348'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:55.993507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e45b31eb-beec-4b9d-850d-08e08cafc348 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59540531d83242cb80c880f2341cfbbc', 'request_id': 'req-e45b31eb-beec-4b9d-850d-08e08cafc348'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:56.002612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e45b31eb-beec-4b9d-850d-08e08cafc348 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59540531d83242cb80c880f2341cfbbc', 'request_id': 'req-e45b31eb-beec-4b9d-850d-08e08cafc348'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:56.005648 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1074/2170] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:56.011121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb53b19-c875-4479-ae81-aeb759f01163 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.012091 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb53b19-c875-4479-ae81-aeb759f01163 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.012091 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb53b19-c875-4479-ae81-aeb759f01163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.021748 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e45b31eb-beec-4b9d-850d-08e08cafc348 admin admin] The token cache is being invalidate because project 59540531d83242cb80c880f2341cfbbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:13:56.022044 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e45b31eb-beec-4b9d-850d-08e08cafc348 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:56.023717 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e45b31eb-beec-4b9d-850d-08e08cafc348 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:13:56.027969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1097/2171] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:13:55 2022] DELETE /identity/v3/projects/59540531d83242cb80c880f2341cfbbc => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:56.029928 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b31eb-beec-4b9d-850d-08e08cafc348 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:56.034894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e45b31eb-beec-4b9d-850d-08e08cafc348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:56.055276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-9fb53b19-c875-4479-ae81-aeb759f01163 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:56.056432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9fb53b19-c875-4479-ae81-aeb759f01163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:56.141733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b31eb-beec-4b9d-850d-08e08cafc348 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:56.147868 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54fe3f-7f1c-4957-9004-b2747df93929 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57a5f04cb9674b86a6b4b9a472e1afc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:56.148663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54fe3f-7f1c-4957-9004-b2747df93929 service glance] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.148931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54fe3f-7f1c-4957-9004-b2747df93929 service glance] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.149267 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54fe3f-7f1c-4957-9004-b2747df93929 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.151904 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-da54fe3f-7f1c-4957-9004-b2747df93929 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:56.185837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:56.195273 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1075/2172] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:56.198024 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5512f71-0b0f-481c-b5c0-9a346084daec None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.198580 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5512f71-0b0f-481c-b5c0-9a346084daec None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.199013 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5512f71-0b0f-481c-b5c0-9a346084daec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.224709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-a5512f71-0b0f-481c-b5c0-9a346084daec None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:56.226080 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-a5512f71-0b0f-481c-b5c0-9a346084daec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:56.231192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54fe3f-7f1c-4957-9004-b2747df93929 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:56.232805 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54fe3f-7f1c-4957-9004-b2747df93929 service glance] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:56.244133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1076/2173] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:56.244133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603663bb-6553-4c3f-b1b6-187d8b172857 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.244133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603663bb-6553-4c3f-b1b6-187d8b172857 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.244133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603663bb-6553-4c3f-b1b6-187d8b172857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.254636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1098/2174] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:56.254636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a664d6c-a851-4c2c-a179-0dc36010c7a5 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.254636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a664d6c-a851-4c2c-a179-0dc36010c7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.254636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a664d6c-a851-4c2c-a179-0dc36010c7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.264280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-603663bb-6553-4c3f-b1b6-187d8b172857 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:56.265872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-603663bb-6553-4c3f-b1b6-187d8b172857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:56.272432 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-6a664d6c-a851-4c2c-a179-0dc36010c7a5 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:56.273572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6a664d6c-a851-4c2c-a179-0dc36010c7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:56.278263 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1077/2175] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:56.279285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603663bb-6553-4c3f-b1b6-187d8b172857 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:56.285188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1099/2176] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:56.289436 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603663bb-6553-4c3f-b1b6-187d8b172857 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:56.294810 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822107c-9670-4e48-8ec7-1547f4261c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:56.295498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822107c-9670-4e48-8ec7-1547f4261c2e admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.295776 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822107c-9670-4e48-8ec7-1547f4261c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.296172 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822107c-9670-4e48-8ec7-1547f4261c2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.297663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a664d6c-a851-4c2c-a179-0dc36010c7a5 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:56.298115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7822107c-9670-4e48-8ec7-1547f4261c2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:56.298115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7822107c-9670-4e48-8ec7-1547f4261c2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:56.298781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7822107c-9670-4e48-8ec7-1547f4261c2e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:56.317835 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1078/2177] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:56.321936 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822107c-9670-4e48-8ec7-1547f4261c2e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:56.331321 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a664d6c-a851-4c2c-a179-0dc36010c7a5 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:56.333970 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822107c-9670-4e48-8ec7-1547f4261c2e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:56.339888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754d4a5-45b2-4f97-8dba-85130ec3d365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:56.341146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754d4a5-45b2-4f97-8dba-85130ec3d365 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.341829 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754d4a5-45b2-4f97-8dba-85130ec3d365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.342187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754d4a5-45b2-4f97-8dba-85130ec3d365 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.342935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754d4a5-45b2-4f97-8dba-85130ec3d365 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:56.344950 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754d4a5-45b2-4f97-8dba-85130ec3d365 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:56.366457 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445752b-629d-4346-82e2-6e215136a19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:56.367008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445752b-629d-4346-82e2-6e215136a19b admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.367289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445752b-629d-4346-82e2-6e215136a19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.367603 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445752b-629d-4346-82e2-6e215136a19b admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.368338 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445752b-629d-4346-82e2-6e215136a19b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:13:56.368682 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445752b-629d-4346-82e2-6e215136a19b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:56.370438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445752b-629d-4346-82e2-6e215136a19b admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:56.386798 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1079/2178] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:56 2022] POST /identity/v3/projects => generated 363 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:56.391956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754d4a5-45b2-4f97-8dba-85130ec3d365 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:56.413994 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1100/2179] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:13:56.418023 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445752b-629d-4346-82e2-6e215136a19b admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:56.435875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445752b-629d-4346-82e2-6e215136a19b admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:56.490534 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754d4a5-45b2-4f97-8dba-85130ec3d365 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:56.495528 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc63d3a-14f2-4f36-9faf-3d9dabe79c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:56.496165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc63d3a-14f2-4f36-9faf-3d9dabe79c67 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.496458 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc63d3a-14f2-4f36-9faf-3d9dabe79c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.496725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc63d3a-14f2-4f36-9faf-3d9dabe79c67 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.497412 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc63d3a-14f2-4f36-9faf-3d9dabe79c67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:56.499393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc63d3a-14f2-4f36-9faf-3d9dabe79c67 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:56.499898 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6c9b2-6576-4112-9ad9-1027b96eb3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:56.500512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6c9b2-6576-4112-9ad9-1027b96eb3c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.500792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6c9b2-6576-4112-9ad9-1027b96eb3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.501064 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6c9b2-6576-4112-9ad9-1027b96eb3c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.501704 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6c9b2-6576-4112-9ad9-1027b96eb3c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:56.503823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6c9b2-6576-4112-9ad9-1027b96eb3c3 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:56.524315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1080/2180] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:56 2022] POST /identity/v3/users => generated 282 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:56.528474 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc63d3a-14f2-4f36-9faf-3d9dabe79c67 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:56.539446 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc63d3a-14f2-4f36-9faf-3d9dabe79c67 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:56.541322 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1101/2181] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:56 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:56.542311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6c9b2-6576-4112-9ad9-1027b96eb3c3 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:56.544788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:56.612046 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c919f-89e8-462f-b900-0ab89baaefa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:56.612670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c919f-89e8-462f-b900-0ab89baaefa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.613104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c919f-89e8-462f-b900-0ab89baaefa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.613547 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c919f-89e8-462f-b900-0ab89baaefa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.614212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c919f-89e8-462f-b900-0ab89baaefa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:56.615882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c919f-89e8-462f-b900-0ab89baaefa0 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:56.616153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6c9b2-6576-4112-9ad9-1027b96eb3c3 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:56.622487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f0fbd-a68d-49bf-b68d-54a60aa0cb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:56.623750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f0fbd-a68d-49bf-b68d-54a60aa0cb37 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.624032 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f0fbd-a68d-49bf-b68d-54a60aa0cb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.624299 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f0fbd-a68d-49bf-b68d-54a60aa0cb37 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.625055 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f0fbd-a68d-49bf-b68d-54a60aa0cb37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:56.627446 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1081/2182] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:56.627574 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f0fbd-a68d-49bf-b68d-54a60aa0cb37 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:56.630293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c919f-89e8-462f-b900-0ab89baaefa0 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:56.643686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c919f-89e8-462f-b900-0ab89baaefa0 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:56.648841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0428a-52e1-496b-bf10-04a643c9a491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:56.649614 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0428a-52e1-496b-bf10-04a643c9a491 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.649901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0428a-52e1-496b-bf10-04a643c9a491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.650176 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0428a-52e1-496b-bf10-04a643c9a491 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.651294 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec0428a-52e1-496b-bf10-04a643c9a491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:56.653013 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec0428a-52e1-496b-bf10-04a643c9a491 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:56.660992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1102/2183] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:56 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:56.665967 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f0fbd-a68d-49bf-b68d-54a60aa0cb37 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:56.677852 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f0fbd-a68d-49bf-b68d-54a60aa0cb37 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:56.682450 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1082/2184] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:56.695964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e044208-9906-4c0d-bcde-9a91629d98e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:56.696944 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e044208-9906-4c0d-bcde-9a91629d98e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.697481 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0428a-52e1-496b-bf10-04a643c9a491 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:56.697723 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e044208-9906-4c0d-bcde-9a91629d98e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.698184 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e044208-9906-4c0d-bcde-9a91629d98e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.698883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e044208-9906-4c0d-bcde-9a91629d98e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:56.702273 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e044208-9906-4c0d-bcde-9a91629d98e3 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:56.721829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1103/2185] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:56.723480 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0428a-52e1-496b-bf10-04a643c9a491 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:56.724104 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e044208-9906-4c0d-bcde-9a91629d98e3 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:56.735401 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6ed09b-d513-4711-aa04-5765e993d6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:56.740186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ed09b-d513-4711-aa04-5765e993d6db admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.740475 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ed09b-d513-4711-aa04-5765e993d6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.740737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ed09b-d513-4711-aa04-5765e993d6db admin admin] PATH_INFO: `/v3/projects/e92331000215401385aa364aa737be0c/users/17ba5f467ba343e0ae5a1a636747e4e2/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.750924 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e044208-9906-4c0d-bcde-9a91629d98e3 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:56.763174 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da3d41-8299-4057-b805-3593588d900c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:56.764004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da3d41-8299-4057-b805-3593588d900c admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.764282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da3d41-8299-4057-b805-3593588d900c admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.764549 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da3d41-8299-4057-b805-3593588d900c admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.765197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da3d41-8299-4057-b805-3593588d900c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:56.767042 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da3d41-8299-4057-b805-3593588d900c admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:56.769990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ed09b-d513-4711-aa04-5765e993d6db admin admin] RBAC: Authorizing `identity:create_grant(project_id=e92331000215401385aa364aa737be0c, user_id=17ba5f467ba343e0ae5a1a636747e4e2, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:56.774780 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:56.775503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ed09b-d513-4711-aa04-5765e993d6db admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:56.785012 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1104/2186] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:56.788987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da3d41-8299-4057-b805-3593588d900c admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:56.812934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da3d41-8299-4057-b805-3593588d900c admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:56.815692 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1083/2187] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:56 2022] PUT /identity/v3/projects/e92331000215401385aa364aa737be0c/users/17ba5f467ba343e0ae5a1a636747e4e2/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:56.827972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235ff5f-39c3-40ae-a80e-4ea61a2b6e50 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.828284 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235ff5f-39c3-40ae-a80e-4ea61a2b6e50 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.829331 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235ff5f-39c3-40ae-a80e-4ea61a2b6e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.911139 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-7235ff5f-39c3-40ae-a80e-4ea61a2b6e50 None None] MFA Rules not processed for user `17ba5f467ba343e0ae5a1a636747e4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:56.915225 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-7235ff5f-39c3-40ae-a80e-4ea61a2b6e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:56.936848 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e14b0-f5d1-484a-b7e1-5be09ff79a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:56.937714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e14b0-f5d1-484a-b7e1-5be09ff79a0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:56.938071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e14b0-f5d1-484a-b7e1-5be09ff79a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:56.938366 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e14b0-f5d1-484a-b7e1-5be09ff79a0f admin admin] PATH_INFO: `/v3/projects/7cf48cd3747b48c3acaaae55885222bd/users/958deb6db20945b785b1970682945e7c/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:56.959930 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e14b0-f5d1-484a-b7e1-5be09ff79a0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cf48cd3747b48c3acaaae55885222bd, user_id=958deb6db20945b785b1970682945e7c, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:56.964474 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e14b0-f5d1-484a-b7e1-5be09ff79a0f admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:56.987675 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1105/2188] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:56 2022] PUT /identity/v3/projects/7cf48cd3747b48c3acaaae55885222bd/users/958deb6db20945b785b1970682945e7c/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:57.000806 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6d323-4bc6-48f5-a790-d81c3c3c1d34 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:57.001214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6d323-4bc6-48f5-a790-d81c3c3c1d34 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:57.001634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6d323-4bc6-48f5-a790-d81c3c3c1d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:57.039451 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-a2c6d323-4bc6-48f5-a790-d81c3c3c1d34 None None] MFA Rules not processed for user `958deb6db20945b785b1970682945e7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:57.041102 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a2c6d323-4bc6-48f5-a790-d81c3c3c1d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:57.052442 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1084/2189] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:57.053775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235ff5f-39c3-40ae-a80e-4ea61a2b6e50 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:57.068576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235ff5f-39c3-40ae-a80e-4ea61a2b6e50 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:57.120313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1106/2190] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:57.126299 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c6d323-4bc6-48f5-a790-d81c3c3c1d34 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:57.136839 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:57.146949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7b85e-83cb-4ac6-8d4a-2cb7e79af905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:57.147851 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7b85e-83cb-4ac6-8d4a-2cb7e79af905 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:57.148526 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7b85e-83cb-4ac6-8d4a-2cb7e79af905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:57.148804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7b85e-83cb-4ac6-8d4a-2cb7e79af905 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:57.149444 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7b85e-83cb-4ac6-8d4a-2cb7e79af905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:57.151857 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7b85e-83cb-4ac6-8d4a-2cb7e79af905 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:57.185847 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1085/2191] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:57 2022] POST /identity/v3/projects => generated 363 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:57.189884 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7b85e-83cb-4ac6-8d4a-2cb7e79af905 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:57.219973 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c6d323-4bc6-48f5-a790-d81c3c3c1d34 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:57.241224 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:57.287397 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7b85e-83cb-4ac6-8d4a-2cb7e79af905 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:57.297759 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2339c-28e6-41e3-9a2d-c7ffbb6e3a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:57.298533 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2339c-28e6-41e3-9a2d-c7ffbb6e3a86 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:57.298927 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2339c-28e6-41e3-9a2d-c7ffbb6e3a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:57.299204 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2339c-28e6-41e3-9a2d-c7ffbb6e3a86 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:57.300044 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2339c-28e6-41e3-9a2d-c7ffbb6e3a86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:57.303356 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2339c-28e6-41e3-9a2d-c7ffbb6e3a86 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:57.306720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca18c75-0df4-4040-b415-7f111edd9109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:57.308694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca18c75-0df4-4040-b415-7f111edd9109 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:57.308977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca18c75-0df4-4040-b415-7f111edd9109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:57.309339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca18c75-0df4-4040-b415-7f111edd9109 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:57.309989 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca18c75-0df4-4040-b415-7f111edd9109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:57.312069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca18c75-0df4-4040-b415-7f111edd9109 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:57.346949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1107/2192] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:57 2022] POST /identity/v3/projects => generated 377 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:57.352039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca18c75-0df4-4040-b415-7f111edd9109 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:57.355447 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1086/2193] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:57 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:57.355610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2339c-28e6-41e3-9a2d-c7ffbb6e3a86 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:57.372864 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2339c-28e6-41e3-9a2d-c7ffbb6e3a86 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:57.443199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca18c75-0df4-4040-b415-7f111edd9109 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:57.454556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c1c7f-5d40-4c5a-ae46-37a0343eea3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:57.455727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c1c7f-5d40-4c5a-ae46-37a0343eea3e admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:57.456214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c1c7f-5d40-4c5a-ae46-37a0343eea3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:57.456696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c1c7f-5d40-4c5a-ae46-37a0343eea3e admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:57.457964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c1c7f-5d40-4c5a-ae46-37a0343eea3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:57.462821 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c1c7f-5d40-4c5a-ae46-37a0343eea3e admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:57.473751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292dcc22-eac2-4c97-9dc2-ae7fe2a9728c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:57.474372 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dcc22-eac2-4c97-9dc2-ae7fe2a9728c admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:57.474646 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dcc22-eac2-4c97-9dc2-ae7fe2a9728c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:57.474944 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dcc22-eac2-4c97-9dc2-ae7fe2a9728c admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:57.475686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292dcc22-eac2-4c97-9dc2-ae7fe2a9728c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:57.477784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292dcc22-eac2-4c97-9dc2-ae7fe2a9728c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:57.501282 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1087/2194] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:57.501282 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292dcc22-eac2-4c97-9dc2-ae7fe2a9728c admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:57.501716 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1108/2195] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:57 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:57.508233 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c1c7f-5d40-4c5a-ae46-37a0343eea3e admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:57.510999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292dcc22-eac2-4c97-9dc2-ae7fe2a9728c admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:57.518967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287d7a6-2707-4562-8f36-d88b9eb92508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:57.520171 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287d7a6-2707-4562-8f36-d88b9eb92508 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:57.520657 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287d7a6-2707-4562-8f36-d88b9eb92508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:57.521156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287d7a6-2707-4562-8f36-d88b9eb92508 admin admin] PATH_INFO: `/v3/projects/17947c0c82ac40f6a0bfb376a154150e/users/e35380e24def4e75852d514ba1e62200/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:57.524609 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c1c7f-5d40-4c5a-ae46-37a0343eea3e admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:57.535796 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916469a-7c54-48b9-b9f2-4f1c9f2429c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:57.536095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916469a-7c54-48b9-b9f2-4f1c9f2429c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:57.536437 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916469a-7c54-48b9-b9f2-4f1c9f2429c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:57.536731 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916469a-7c54-48b9-b9f2-4f1c9f2429c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:57.538455 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916469a-7c54-48b9-b9f2-4f1c9f2429c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:57.540824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916469a-7c54-48b9-b9f2-4f1c9f2429c7 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:57.547527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287d7a6-2707-4562-8f36-d88b9eb92508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17947c0c82ac40f6a0bfb376a154150e, user_id=e35380e24def4e75852d514ba1e62200, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:57.551540 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287d7a6-2707-4562-8f36-d88b9eb92508 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:57.556526 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1109/2196] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:57.574636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916469a-7c54-48b9-b9f2-4f1c9f2429c7 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:57.575221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1088/2197] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:57 2022] PUT /identity/v3/projects/17947c0c82ac40f6a0bfb376a154150e/users/e35380e24def4e75852d514ba1e62200/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:57.588080 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287d7a6-2707-4562-8f36-d88b9eb92508 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:57.590829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916469a-7c54-48b9-b9f2-4f1c9f2429c7 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:57.658069 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:57.692773 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287d7a6-2707-4562-8f36-d88b9eb92508 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:57.711552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d43506-10f5-4ed8-b774-d08dd5ff2a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:57.712190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d43506-10f5-4ed8-b774-d08dd5ff2a09 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:57.712464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d43506-10f5-4ed8-b774-d08dd5ff2a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:57.712824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d43506-10f5-4ed8-b774-d08dd5ff2a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:57.713281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc83d7-3773-46f7-ab24-cd30bbdad8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:57.713489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d43506-10f5-4ed8-b774-d08dd5ff2a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:57.714450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc83d7-3773-46f7-ab24-cd30bbdad8bc admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:57.715301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d43506-10f5-4ed8-b774-d08dd5ff2a09 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:57.715533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc83d7-3773-46f7-ab24-cd30bbdad8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:57.716092 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc83d7-3773-46f7-ab24-cd30bbdad8bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:57.717200 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc83d7-3773-46f7-ab24-cd30bbdad8bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:57.719948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc83d7-3773-46f7-ab24-cd30bbdad8bc admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:57.733665 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1089/2198] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:13:57.734805 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d43506-10f5-4ed8-b774-d08dd5ff2a09 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:57.748562 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d43506-10f5-4ed8-b774-d08dd5ff2a09 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:57.752677 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1110/2199] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:57.753733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc83d7-3773-46f7-ab24-cd30bbdad8bc admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:57.753851 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592e8b7-3244-41d1-a46d-3b759da9e479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:57.754507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592e8b7-3244-41d1-a46d-3b759da9e479 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:57.754792 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592e8b7-3244-41d1-a46d-3b759da9e479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:57.755050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592e8b7-3244-41d1-a46d-3b759da9e479 admin admin] PATH_INFO: `/v3/projects/17947c0c82ac40f6a0bfb376a154150e/users/e35380e24def4e75852d514ba1e62200/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:57.767923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f592e8b7-3244-41d1-a46d-3b759da9e479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17947c0c82ac40f6a0bfb376a154150e, user_id=e35380e24def4e75852d514ba1e62200, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:57.772646 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f592e8b7-3244-41d1-a46d-3b759da9e479 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:57.776759 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc83d7-3773-46f7-ab24-cd30bbdad8bc admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:57.786746 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c7f24-f583-4a1f-9ff1-48fd72ef9024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:57.787476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c7f24-f583-4a1f-9ff1-48fd72ef9024 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:57.787757 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c7f24-f583-4a1f-9ff1-48fd72ef9024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:57.788155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c7f24-f583-4a1f-9ff1-48fd72ef9024 admin admin] PATH_INFO: `/v3/projects/c8bc3cbe55f041d6a39e0583134170c5/users/4f3905476438460da404d8c86c8e75ba/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:57.796627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1090/2200] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:57 2022] PUT /identity/v3/projects/17947c0c82ac40f6a0bfb376a154150e/users/e35380e24def4e75852d514ba1e62200/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:57.809953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48382477-e2c2-4496-a4a7-11d8020d782a None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:57.810268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48382477-e2c2-4496-a4a7-11d8020d782a None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:57.811002 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48382477-e2c2-4496-a4a7-11d8020d782a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:57.837447 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c7f24-f583-4a1f-9ff1-48fd72ef9024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8bc3cbe55f041d6a39e0583134170c5, user_id=4f3905476438460da404d8c86c8e75ba, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:57.840796 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c7f24-f583-4a1f-9ff1-48fd72ef9024 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:57.858699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:57.862381 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-48382477-e2c2-4496-a4a7-11d8020d782a None None] MFA Rules not processed for user `e35380e24def4e75852d514ba1e62200`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:57.864174 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-48382477-e2c2-4496-a4a7-11d8020d782a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:57.877059 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1111/2201] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:57 2022] PUT /identity/v3/projects/c8bc3cbe55f041d6a39e0583134170c5/users/4f3905476438460da404d8c86c8e75ba/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:13:57.887942 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840364ee-d233-4e7d-9442-8941d52dfdc1 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:57.888223 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840364ee-d233-4e7d-9442-8941d52dfdc1 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:57.888506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840364ee-d233-4e7d-9442-8941d52dfdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:57.925803 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-840364ee-d233-4e7d-9442-8941d52dfdc1 None None] MFA Rules not processed for user `4f3905476438460da404d8c86c8e75ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:57.927103 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-840364ee-d233-4e7d-9442-8941d52dfdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:58.029239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1112/2202] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:13:58.033158 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840364ee-d233-4e7d-9442-8941d52dfdc1 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:58.033422 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1091/2203] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:58.069903 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48382477-e2c2-4496-a4a7-11d8020d782a None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:58.072773 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-48382477-e2c2-4496-a4a7-11d8020d782a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:58.201470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48382477-e2c2-4496-a4a7-11d8020d782a None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:58.207835 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840364ee-d233-4e7d-9442-8941d52dfdc1 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:58.215290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32381dfa-1c66-4794-ab56-fc3c6f8b2f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:58.216165 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32381dfa-1c66-4794-ab56-fc3c6f8b2f3d admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:58.216497 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32381dfa-1c66-4794-ab56-fc3c6f8b2f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:58.216757 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32381dfa-1c66-4794-ab56-fc3c6f8b2f3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:58.219598 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32381dfa-1c66-4794-ab56-fc3c6f8b2f3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:58.222634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32381dfa-1c66-4794-ab56-fc3c6f8b2f3d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:58.262497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9982ab-e693-4ab7-916d-84e44c273e70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:58.262974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9982ab-e693-4ab7-916d-84e44c273e70 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:58.263273 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9982ab-e693-4ab7-916d-84e44c273e70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:58.264106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9982ab-e693-4ab7-916d-84e44c273e70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:58.264219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1113/2204] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:13:58 2022] POST /identity/v3/projects => generated 373 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:13:58.265189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32381dfa-1c66-4794-ab56-fc3c6f8b2f3d admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:58.289745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:58.331846 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32381dfa-1c66-4794-ab56-fc3c6f8b2f3d admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:58.334463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9982ab-e693-4ab7-916d-84e44c273e70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:58.335966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9982ab-e693-4ab7-916d-84e44c273e70 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:58.337417 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904e90f-36cc-468b-8fc1-15546a38c803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:58.338035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904e90f-36cc-468b-8fc1-15546a38c803 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:58.338311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904e90f-36cc-468b-8fc1-15546a38c803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:58.338580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904e90f-36cc-468b-8fc1-15546a38c803 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:58.339214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904e90f-36cc-468b-8fc1-15546a38c803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:58.341183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904e90f-36cc-468b-8fc1-15546a38c803 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:58.346636 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1092/2205] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:58.361056 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:58.372182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1114/2206] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:13:58 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:13:58.377728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9982ab-e693-4ab7-916d-84e44c273e70 service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:58.396665 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9982ab-e693-4ab7-916d-84e44c273e70 service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:58.401745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b05f4-f0ec-4a5d-bf72-b6655aaef63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:58.402428 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b05f4-f0ec-4a5d-bf72-b6655aaef63d admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:58.402705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b05f4-f0ec-4a5d-bf72-b6655aaef63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:58.402978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b05f4-f0ec-4a5d-bf72-b6655aaef63d admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:58.403792 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b05f4-f0ec-4a5d-bf72-b6655aaef63d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:58.405497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b05f4-f0ec-4a5d-bf72-b6655aaef63d admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:58.419524 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1093/2207] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:58.421160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904e90f-36cc-468b-8fc1-15546a38c803 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:58.431908 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904e90f-36cc-468b-8fc1-15546a38c803 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:58.437684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac29a51-0569-45cf-a8ca-f7ab52a8333b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:58.438332 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac29a51-0569-45cf-a8ca-f7ab52a8333b admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:58.438609 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac29a51-0569-45cf-a8ca-f7ab52a8333b admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:58.438885 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac29a51-0569-45cf-a8ca-f7ab52a8333b admin admin] PATH_INFO: `/v3/projects/37fbb504c8464bc0871f6b9cbbe6c545/users/cbfad94c7dad4a9184aa30d81876abd1/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:58.443383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b05f4-f0ec-4a5d-bf72-b6655aaef63d admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:58.458438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac29a51-0569-45cf-a8ca-f7ab52a8333b admin admin] RBAC: Authorizing `identity:create_grant(project_id=37fbb504c8464bc0871f6b9cbbe6c545, user_id=cbfad94c7dad4a9184aa30d81876abd1, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:58.461938 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac29a51-0569-45cf-a8ca-f7ab52a8333b admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:58.478208 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1115/2208] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:58 2022] PUT /identity/v3/projects/37fbb504c8464bc0871f6b9cbbe6c545/users/cbfad94c7dad4a9184aa30d81876abd1/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:58.482422 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac29a51-0569-45cf-a8ca-f7ab52a8333b admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:58.521826 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b05f4-f0ec-4a5d-bf72-b6655aaef63d admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:58.555164 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac29a51-0569-45cf-a8ca-f7ab52a8333b admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:58.560219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40aac0-c7fb-4f1f-8068-42930598bb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:58.560825 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40aac0-c7fb-4f1f-8068-42930598bb7d admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:58.561382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40aac0-c7fb-4f1f-8068-42930598bb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:58.561382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40aac0-c7fb-4f1f-8068-42930598bb7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:58.561990 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40aac0-c7fb-4f1f-8068-42930598bb7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:58.563628 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40aac0-c7fb-4f1f-8068-42930598bb7d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:58.574727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1116/2209] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:13:58.578715 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40aac0-c7fb-4f1f-8068-42930598bb7d admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:58.586672 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be76ccd-6b09-4c65-acae-1aa7741723b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:58.587384 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be76ccd-6b09-4c65-acae-1aa7741723b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:58.587778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be76ccd-6b09-4c65-acae-1aa7741723b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:58.588058 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be76ccd-6b09-4c65-acae-1aa7741723b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:58.590623 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40aac0-c7fb-4f1f-8068-42930598bb7d admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:58.595708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511b218-4e9b-417c-b50d-0d5afadd14ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:58.596360 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511b218-4e9b-417c-b50d-0d5afadd14ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:58.596701 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511b218-4e9b-417c-b50d-0d5afadd14ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:58.596960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511b218-4e9b-417c-b50d-0d5afadd14ba admin admin] PATH_INFO: `/v3/projects/37fbb504c8464bc0871f6b9cbbe6c545/users/cbfad94c7dad4a9184aa30d81876abd1/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:58.599318 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511b218-4e9b-417c-b50d-0d5afadd14ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=37fbb504c8464bc0871f6b9cbbe6c545, user_id=cbfad94c7dad4a9184aa30d81876abd1, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:58.602232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511b218-4e9b-417c-b50d-0d5afadd14ba admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:58.623526 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1117/2210] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:13:58 2022] PUT /identity/v3/projects/37fbb504c8464bc0871f6b9cbbe6c545/users/cbfad94c7dad4a9184aa30d81876abd1/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:13:58.627260 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c7ab3-258d-454d-b188-02b50d9c2f08 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:58.627629 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c7ab3-258d-454d-b188-02b50d9c2f08 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:58.627967 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c7ab3-258d-454d-b188-02b50d9c2f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:58.672624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be76ccd-6b09-4c65-acae-1aa7741723b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:58.675739 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be76ccd-6b09-4c65-acae-1aa7741723b7 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:58.680224 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-a09c7ab3-258d-454d-b188-02b50d9c2f08 None None] MFA Rules not processed for user `cbfad94c7dad4a9184aa30d81876abd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:13:58.695764 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a09c7ab3-258d-454d-b188-02b50d9c2f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:58.697019 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1094/2211] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:13:58.811197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1118/2212] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:13:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:13:58.824830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be76ccd-6b09-4c65-acae-1aa7741723b7 service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:13:58.826392 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-7be76ccd-6b09-4c65-acae-1aa7741723b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:13:58.842399 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be76ccd-6b09-4c65-acae-1aa7741723b7 service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:13:58.865560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:13:58.918477 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17ea1f-729e-476b-9d26-db499070debb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:13:58.919111 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17ea1f-729e-476b-9d26-db499070debb service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:13:58.919386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17ea1f-729e-476b-9d26-db499070debb service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:13:58.919695 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17ea1f-729e-476b-9d26-db499070debb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:13:58.980906 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17ea1f-729e-476b-9d26-db499070debb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:13:58.982572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17ea1f-729e-476b-9d26-db499070debb service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:13:58.993109 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1095/2213] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:13:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:14:00.688916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c7ab3-258d-454d-b188-02b50d9c2f08 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:00.698498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c7ab3-258d-454d-b188-02b50d9c2f08 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:00.702279 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e227f-827c-45fd-a1d4-918baeaab223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:00.702882 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e227f-827c-45fd-a1d4-918baeaab223 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:00.703159 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e227f-827c-45fd-a1d4-918baeaab223 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:00.703517 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e227f-827c-45fd-a1d4-918baeaab223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:00.748258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:00.765627 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e227f-827c-45fd-a1d4-918baeaab223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:00.767137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e227f-827c-45fd-a1d4-918baeaab223 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:00.777925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1119/2214] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:14:01.363995 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17ea1f-729e-476b-9d26-db499070debb service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:01.374306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17ea1f-729e-476b-9d26-db499070debb service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:01.378452 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594e9f2-9b9b-4f6a-95cb-db7581448337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:01.379098 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594e9f2-9b9b-4f6a-95cb-db7581448337 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:01.379376 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594e9f2-9b9b-4f6a-95cb-db7581448337 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:01.379702 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594e9f2-9b9b-4f6a-95cb-db7581448337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:01.390316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594e9f2-9b9b-4f6a-95cb-db7581448337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:01.391853 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594e9f2-9b9b-4f6a-95cb-db7581448337 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:01.403634 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1096/2215] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:14:01.665964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e227f-827c-45fd-a1d4-918baeaab223 service cinder] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:01.758608 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e227f-827c-45fd-a1d4-918baeaab223 service cinder] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:01.764950 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ae1a3-45d2-4574-9135-1781024f3986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:01.766007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ae1a3-45d2-4574-9135-1781024f3986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:01.766490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ae1a3-45d2-4574-9135-1781024f3986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:01.766945 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ae1a3-45d2-4574-9135-1781024f3986 admin admin] PATH_INFO: `/v3/users/17ba5f467ba343e0ae5a1a636747e4e2` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:01.768340 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ae1a3-45d2-4574-9135-1781024f3986 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17ba5f467ba343e0ae5a1a636747e4e2)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:01.772436 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ae1a3-45d2-4574-9135-1781024f3986 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:01.850394 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f74ae1a3-45d2-4574-9135-1781024f3986 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17ba5f467ba343e0ae5a1a636747e4e2'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:01.851253 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f74ae1a3-45d2-4574-9135-1781024f3986 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17ba5f467ba343e0ae5a1a636747e4e2'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:01.852115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f74ae1a3-45d2-4574-9135-1781024f3986 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17ba5f467ba343e0ae5a1a636747e4e2'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:01.862978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f74ae1a3-45d2-4574-9135-1781024f3986 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17ba5f467ba343e0ae5a1a636747e4e2'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:01.877094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f74ae1a3-45d2-4574-9135-1781024f3986 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17ba5f467ba343e0ae5a1a636747e4e2'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:01.893028 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1120/2216] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:14:01 2022] DELETE /identity/v3/users/17ba5f467ba343e0ae5a1a636747e4e2 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:01.898541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683b973-9308-4efe-b2f3-7e45e90d2cc3 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:01.898816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683b973-9308-4efe-b2f3-7e45e90d2cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:01.899099 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683b973-9308-4efe-b2f3-7e45e90d2cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:01.922437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-a683b973-9308-4efe-b2f3-7e45e90d2cc3 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:01.923877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-a683b973-9308-4efe-b2f3-7e45e90d2cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:02.017604 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1097/2217] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:14:02.028658 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ae1a3-45d2-4574-9135-1781024f3986 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:02.030199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-f74ae1a3-45d2-4574-9135-1781024f3986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:02.119951 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ae1a3-45d2-4574-9135-1781024f3986 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:02.124567 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420f4eb-08ef-46e3-83fc-acf5f8a0d065 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:02.125192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420f4eb-08ef-46e3-83fc-acf5f8a0d065 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:02.125433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420f4eb-08ef-46e3-83fc-acf5f8a0d065 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:02.125706 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420f4eb-08ef-46e3-83fc-acf5f8a0d065 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:02.138996 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420f4eb-08ef-46e3-83fc-acf5f8a0d065 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:02.141194 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420f4eb-08ef-46e3-83fc-acf5f8a0d065 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:02.153746 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1121/2218] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:14:02.493924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683b973-9308-4efe-b2f3-7e45e90d2cc3 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:02.495833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-a683b973-9308-4efe-b2f3-7e45e90d2cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:02.511926 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683b973-9308-4efe-b2f3-7e45e90d2cc3 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:02.519998 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d325821-2b80-4e6d-b090-0991cc0e8dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:02.520688 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d325821-2b80-4e6d-b090-0991cc0e8dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:02.520965 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d325821-2b80-4e6d-b090-0991cc0e8dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:02.521233 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d325821-2b80-4e6d-b090-0991cc0e8dfb admin admin] PATH_INFO: `/v3/projects/e92331000215401385aa364aa737be0c` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:02.522771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d325821-2b80-4e6d-b090-0991cc0e8dfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e92331000215401385aa364aa737be0c)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:02.525050 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d325821-2b80-4e6d-b090-0991cc0e8dfb admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:02.605021 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-2d325821-2b80-4e6d-b090-0991cc0e8dfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e92331000215401385aa364aa737be0c', 'request_id': 'req-2d325821-2b80-4e6d-b090-0991cc0e8dfb'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:02.612905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:02.616373 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-2d325821-2b80-4e6d-b090-0991cc0e8dfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e92331000215401385aa364aa737be0c', 'request_id': 'req-2d325821-2b80-4e6d-b090-0991cc0e8dfb'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:02.628431 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-2d325821-2b80-4e6d-b090-0991cc0e8dfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e92331000215401385aa364aa737be0c', 'request_id': 'req-2d325821-2b80-4e6d-b090-0991cc0e8dfb'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:02.656901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-2d325821-2b80-4e6d-b090-0991cc0e8dfb admin admin] The token cache is being invalidate because project e92331000215401385aa364aa737be0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:14:02.656901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-2d325821-2b80-4e6d-b090-0991cc0e8dfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:02.656901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-2d325821-2b80-4e6d-b090-0991cc0e8dfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:02.656901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1098/2219] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:14:02 2022] DELETE /identity/v3/projects/e92331000215401385aa364aa737be0c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:02.660257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420f4eb-08ef-46e3-83fc-acf5f8a0d065 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:02.669234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:02.674606 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420f4eb-08ef-46e3-83fc-acf5f8a0d065 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:02.778733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e4559-db5a-40fe-927e-1accb6f2f333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:02.782055 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e4559-db5a-40fe-927e-1accb6f2f333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:02.782518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e4559-db5a-40fe-927e-1accb6f2f333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:02.782973 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e4559-db5a-40fe-927e-1accb6f2f333 admin admin] PATH_INFO: `/v3/users/e35380e24def4e75852d514ba1e62200` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:02.785022 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e4559-db5a-40fe-927e-1accb6f2f333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e35380e24def4e75852d514ba1e62200)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:02.788593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e4559-db5a-40fe-927e-1accb6f2f333 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:02.863874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-655e4559-db5a-40fe-927e-1accb6f2f333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e35380e24def4e75852d514ba1e62200'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:02.864751 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-655e4559-db5a-40fe-927e-1accb6f2f333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e35380e24def4e75852d514ba1e62200'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:02.867386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-655e4559-db5a-40fe-927e-1accb6f2f333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e35380e24def4e75852d514ba1e62200'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:02.905922 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-655e4559-db5a-40fe-927e-1accb6f2f333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e35380e24def4e75852d514ba1e62200'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:02.924437 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-655e4559-db5a-40fe-927e-1accb6f2f333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e35380e24def4e75852d514ba1e62200'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:02.954371 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1122/2220] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:14:02 2022] DELETE /identity/v3/users/e35380e24def4e75852d514ba1e62200 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:03.336755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d325821-2b80-4e6d-b090-0991cc0e8dfb admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:03.338352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2d325821-2b80-4e6d-b090-0991cc0e8dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:03.375296 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d325821-2b80-4e6d-b090-0991cc0e8dfb admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:03.452986 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df126adb-928d-49f5-a28a-2ce52f8726fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:03.453647 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df126adb-928d-49f5-a28a-2ce52f8726fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:03.453924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df126adb-928d-49f5-a28a-2ce52f8726fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:03.454197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df126adb-928d-49f5-a28a-2ce52f8726fe admin admin] PATH_INFO: `/v3/projects/17947c0c82ac40f6a0bfb376a154150e` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:03.455386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df126adb-928d-49f5-a28a-2ce52f8726fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=17947c0c82ac40f6a0bfb376a154150e)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:03.457756 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df126adb-928d-49f5-a28a-2ce52f8726fe admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:03.570146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-df126adb-928d-49f5-a28a-2ce52f8726fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17947c0c82ac40f6a0bfb376a154150e', 'request_id': 'req-df126adb-928d-49f5-a28a-2ce52f8726fe'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:03.576894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-df126adb-928d-49f5-a28a-2ce52f8726fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17947c0c82ac40f6a0bfb376a154150e', 'request_id': 'req-df126adb-928d-49f5-a28a-2ce52f8726fe'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:03.591086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-df126adb-928d-49f5-a28a-2ce52f8726fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17947c0c82ac40f6a0bfb376a154150e', 'request_id': 'req-df126adb-928d-49f5-a28a-2ce52f8726fe'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:03.614083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-df126adb-928d-49f5-a28a-2ce52f8726fe admin admin] The token cache is being invalidate because project 17947c0c82ac40f6a0bfb376a154150e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:14:03.614379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-df126adb-928d-49f5-a28a-2ce52f8726fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:03.615246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-df126adb-928d-49f5-a28a-2ce52f8726fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:03.617550 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1099/2221] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:14:03 2022] DELETE /identity/v3/projects/17947c0c82ac40f6a0bfb376a154150e => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:03.622188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b55cd-c9cd-4338-ba90-20e8899b4090 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:03.622445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b55cd-c9cd-4338-ba90-20e8899b4090 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:03.622722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b55cd-c9cd-4338-ba90-20e8899b4090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:03.643910 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-a35b55cd-c9cd-4338-ba90-20e8899b4090 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:03.645151 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a35b55cd-c9cd-4338-ba90-20e8899b4090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:03.724616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1123/2222] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:03.733970 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cadd64-f6c0-4392-8fb8-1903ff64545b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:03.734434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cadd64-f6c0-4392-8fb8-1903ff64545b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:03.734899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cadd64-f6c0-4392-8fb8-1903ff64545b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:03.760870 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-d4cadd64-f6c0-4392-8fb8-1903ff64545b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:03.762101 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d4cadd64-f6c0-4392-8fb8-1903ff64545b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:03.772422 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1100/2223] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:03.777080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b55cd-c9cd-4338-ba90-20e8899b4090 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:03.789593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b55cd-c9cd-4338-ba90-20e8899b4090 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:03.795111 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dca884-dc64-40f6-8e8c-98ca2cac250b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:03.795773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dca884-dc64-40f6-8e8c-98ca2cac250b admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:03.796052 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dca884-dc64-40f6-8e8c-98ca2cac250b admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:03.796329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dca884-dc64-40f6-8e8c-98ca2cac250b admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:03.797022 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dca884-dc64-40f6-8e8c-98ca2cac250b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:03.797373 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dca884-dc64-40f6-8e8c-98ca2cac250b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:03.799143 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dca884-dc64-40f6-8e8c-98ca2cac250b admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:03.812795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1124/2224] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:03.817348 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cadd64-f6c0-4392-8fb8-1903ff64545b None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:03.827316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cadd64-f6c0-4392-8fb8-1903ff64545b None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:03.833210 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a09f9-6bb2-42f4-92fc-882f4f90e7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:03.833912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a09f9-6bb2-42f4-92fc-882f4f90e7cf admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:03.836154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a09f9-6bb2-42f4-92fc-882f4f90e7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:03.836700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a09f9-6bb2-42f4-92fc-882f4f90e7cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:03.837694 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a09f9-6bb2-42f4-92fc-882f4f90e7cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:03.839786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a09f9-6bb2-42f4-92fc-882f4f90e7cf admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:03.872443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1101/2225] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:14:03 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:03.877624 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dca884-dc64-40f6-8e8c-98ca2cac250b admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:03.893587 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dca884-dc64-40f6-8e8c-98ca2cac250b admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:03.901854 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:03.966073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611633b-b348-4bb2-9576-a4e5a97997b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:03.966696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611633b-b348-4bb2-9576-a4e5a97997b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:03.966970 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611633b-b348-4bb2-9576-a4e5a97997b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:03.967237 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611633b-b348-4bb2-9576-a4e5a97997b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:03.967899 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611633b-b348-4bb2-9576-a4e5a97997b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:03.969942 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611633b-b348-4bb2-9576-a4e5a97997b9 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:03.996747 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1125/2226] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:14:03 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:04.001053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a09f9-6bb2-42f4-92fc-882f4f90e7cf admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:04.012541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a09f9-6bb2-42f4-92fc-882f4f90e7cf admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:04.021678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393c20a-e77d-4095-8f91-73445c9e6beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:04.022305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393c20a-e77d-4095-8f91-73445c9e6beb admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:04.022581 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393c20a-e77d-4095-8f91-73445c9e6beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:04.022848 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393c20a-e77d-4095-8f91-73445c9e6beb admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:04.023578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393c20a-e77d-4095-8f91-73445c9e6beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:04.025201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393c20a-e77d-4095-8f91-73445c9e6beb admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:04.036058 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1102/2227] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:04.040398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611633b-b348-4bb2-9576-a4e5a97997b9 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:04.055796 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611633b-b348-4bb2-9576-a4e5a97997b9 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:04.065068 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bff1e-30b3-4a4c-bb88-f4e9a451adce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:04.065751 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bff1e-30b3-4a4c-bb88-f4e9a451adce admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:04.066085 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bff1e-30b3-4a4c-bb88-f4e9a451adce admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:04.066534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bff1e-30b3-4a4c-bb88-f4e9a451adce admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:04.067369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47bff1e-30b3-4a4c-bb88-f4e9a451adce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:04.069559 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47bff1e-30b3-4a4c-bb88-f4e9a451adce admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:04.083456 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1126/2228] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:14:04.087783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393c20a-e77d-4095-8f91-73445c9e6beb admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:04.104360 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393c20a-e77d-4095-8f91-73445c9e6beb admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:04.110361 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7cdd7-79d7-489e-a6c2-1f540efe6447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:04.111149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7cdd7-79d7-489e-a6c2-1f540efe6447 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:04.111561 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7cdd7-79d7-489e-a6c2-1f540efe6447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:04.111931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7cdd7-79d7-489e-a6c2-1f540efe6447 admin admin] PATH_INFO: `/v3/projects/5eaa0ee2fa4f46b7829492ec3bd45e00/users/48f2286950af42ad89f5c9167d699ddd/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:04.134120 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7cdd7-79d7-489e-a6c2-1f540efe6447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eaa0ee2fa4f46b7829492ec3bd45e00, user_id=48f2286950af42ad89f5c9167d699ddd, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:04.138053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7cdd7-79d7-489e-a6c2-1f540efe6447 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:04.168500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1103/2229] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:14:04 2022] PUT /identity/v3/projects/5eaa0ee2fa4f46b7829492ec3bd45e00/users/48f2286950af42ad89f5c9167d699ddd/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:04.179796 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ff7f2-f09e-497c-a500-3093d7a97bd4 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:04.180128 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ff7f2-f09e-497c-a500-3093d7a97bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:04.180467 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ff7f2-f09e-497c-a500-3093d7a97bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:04.219579 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-d28ff7f2-f09e-497c-a500-3093d7a97bd4 None None] MFA Rules not processed for user `48f2286950af42ad89f5c9167d699ddd`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:04.220961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d28ff7f2-f09e-497c-a500-3093d7a97bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:04.327942 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1127/2230] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:14:04.332791 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7cdd7-79d7-489e-a6c2-1f540efe6447 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:04.346438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:04.417656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7cdd7-79d7-489e-a6c2-1f540efe6447 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:04.422752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521f7be-9013-4542-ad11-eb14712cf545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:04.423588 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521f7be-9013-4542-ad11-eb14712cf545 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:04.423870 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521f7be-9013-4542-ad11-eb14712cf545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:04.424147 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521f7be-9013-4542-ad11-eb14712cf545 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:04.424794 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521f7be-9013-4542-ad11-eb14712cf545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:04.427199 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521f7be-9013-4542-ad11-eb14712cf545 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:04.462121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1104/2231] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:14:04 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:04.466476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ff7f2-f09e-497c-a500-3093d7a97bd4 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:04.480028 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ff7f2-f09e-497c-a500-3093d7a97bd4 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:04.544122 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5882ea-c543-430b-8c69-48c96c006a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:04.544774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5882ea-c543-430b-8c69-48c96c006a23 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:04.545252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5882ea-c543-430b-8c69-48c96c006a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:04.545594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5882ea-c543-430b-8c69-48c96c006a23 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:04.546290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5882ea-c543-430b-8c69-48c96c006a23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:04.548596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5882ea-c543-430b-8c69-48c96c006a23 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:04.589541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1128/2232] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:14:04 2022] POST /identity/v3/users => generated 300 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:04.593846 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521f7be-9013-4542-ad11-eb14712cf545 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:04.605866 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521f7be-9013-4542-ad11-eb14712cf545 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:04.611266 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0529f2-123c-489e-adf8-fd9396899f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:04.611918 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0529f2-123c-489e-adf8-fd9396899f5e admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:04.612195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0529f2-123c-489e-adf8-fd9396899f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:04.612463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0529f2-123c-489e-adf8-fd9396899f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:04.613106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0529f2-123c-489e-adf8-fd9396899f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:04.614742 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0529f2-123c-489e-adf8-fd9396899f5e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:04.625629 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1105/2233] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:04.629867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5882ea-c543-430b-8c69-48c96c006a23 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:04.639907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5882ea-c543-430b-8c69-48c96c006a23 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:04.644441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3800f5f0-9c68-48e3-8126-da75985ba519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:04.645086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3800f5f0-9c68-48e3-8126-da75985ba519 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:04.645357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3800f5f0-9c68-48e3-8126-da75985ba519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:04.645664 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3800f5f0-9c68-48e3-8126-da75985ba519 admin admin] PATH_INFO: `/v3/projects/bb2975fe3d5449cdaee6e9ef6138f5e4/users/646183bae905447eb9b3f6ccb948a1a4/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:04.648838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:04.661386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3800f5f0-9c68-48e3-8126-da75985ba519 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb2975fe3d5449cdaee6e9ef6138f5e4, user_id=646183bae905447eb9b3f6ccb948a1a4, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:04.664539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3800f5f0-9c68-48e3-8126-da75985ba519 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:04.678419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1129/2234] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:14:04 2022] PUT /identity/v3/projects/bb2975fe3d5449cdaee6e9ef6138f5e4/users/646183bae905447eb9b3f6ccb948a1a4/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:04.682559 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0529f2-123c-489e-adf8-fd9396899f5e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:04.692817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0529f2-123c-489e-adf8-fd9396899f5e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:04.756783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24648e9-ccd9-454c-8054-630090edc706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:04.757385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24648e9-ccd9-454c-8054-630090edc706 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:04.757658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24648e9-ccd9-454c-8054-630090edc706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:04.757925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24648e9-ccd9-454c-8054-630090edc706 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:04.758551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24648e9-ccd9-454c-8054-630090edc706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:04.760333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24648e9-ccd9-454c-8054-630090edc706 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:04.770145 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1106/2235] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:14:04.774904 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3800f5f0-9c68-48e3-8126-da75985ba519 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:04.785938 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3800f5f0-9c68-48e3-8126-da75985ba519 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:04.791145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ce61e-419c-4765-86a2-6ed282a8adb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:04.791829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ce61e-419c-4765-86a2-6ed282a8adb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:04.792093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ce61e-419c-4765-86a2-6ed282a8adb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:04.792338 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ce61e-419c-4765-86a2-6ed282a8adb7 admin admin] PATH_INFO: `/v3/projects/bb2975fe3d5449cdaee6e9ef6138f5e4/users/646183bae905447eb9b3f6ccb948a1a4/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:04.794290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ce61e-419c-4765-86a2-6ed282a8adb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb2975fe3d5449cdaee6e9ef6138f5e4, user_id=646183bae905447eb9b3f6ccb948a1a4, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:04.796992 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ce61e-419c-4765-86a2-6ed282a8adb7 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:04.808754 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1130/2236] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:14:04 2022] PUT /identity/v3/projects/bb2975fe3d5449cdaee6e9ef6138f5e4/users/646183bae905447eb9b3f6ccb948a1a4/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:04.820410 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcab5f7-7c37-443c-875b-1a0de8d3a08a None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:04.820635 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcab5f7-7c37-443c-875b-1a0de8d3a08a None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:04.820913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcab5f7-7c37-443c-875b-1a0de8d3a08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:04.851187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-ddcab5f7-7c37-443c-875b-1a0de8d3a08a None None] MFA Rules not processed for user `646183bae905447eb9b3f6ccb948a1a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:04.852471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ddcab5f7-7c37-443c-875b-1a0de8d3a08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:04.926295 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1107/2237] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:04.940892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ce61e-419c-4765-86a2-6ed282a8adb7 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:04.942766 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d63ce61e-419c-4765-86a2-6ed282a8adb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:05.003323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ce61e-419c-4765-86a2-6ed282a8adb7 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:05.007376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2b4fc-8287-4168-81c8-892f6e1fb746 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:05.008004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2b4fc-8287-4168-81c8-892f6e1fb746 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:05.008279 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2b4fc-8287-4168-81c8-892f6e1fb746 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:05.008549 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2b4fc-8287-4168-81c8-892f6e1fb746 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:05.018056 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b2b4fc-8287-4168-81c8-892f6e1fb746 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:05.019525 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b2b4fc-8287-4168-81c8-892f6e1fb746 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:05.029133 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1131/2238] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:14:05.107847 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcab5f7-7c37-443c-875b-1a0de8d3a08a None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:05.114999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:05.182260 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcab5f7-7c37-443c-875b-1a0de8d3a08a None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:05.188051 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3e6aa-cb3b-4989-871e-687b54766263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:05.188677 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3e6aa-cb3b-4989-871e-687b54766263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:05.188979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3e6aa-cb3b-4989-871e-687b54766263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:05.189247 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3e6aa-cb3b-4989-871e-687b54766263 admin admin] PATH_INFO: `/v3/users/48f2286950af42ad89f5c9167d699ddd` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:05.190463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3e6aa-cb3b-4989-871e-687b54766263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48f2286950af42ad89f5c9167d699ddd)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:05.192800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3e6aa-cb3b-4989-871e-687b54766263 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:05.251750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-eaf3e6aa-cb3b-4989-871e-687b54766263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48f2286950af42ad89f5c9167d699ddd'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:05.252896 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-eaf3e6aa-cb3b-4989-871e-687b54766263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48f2286950af42ad89f5c9167d699ddd'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:05.253946 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-eaf3e6aa-cb3b-4989-871e-687b54766263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48f2286950af42ad89f5c9167d699ddd'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:05.264005 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-eaf3e6aa-cb3b-4989-871e-687b54766263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48f2286950af42ad89f5c9167d699ddd'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:05.278130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-eaf3e6aa-cb3b-4989-871e-687b54766263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48f2286950af42ad89f5c9167d699ddd'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:05.292994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1108/2239] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:14:05 2022] DELETE /identity/v3/users/48f2286950af42ad89f5c9167d699ddd => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:05.298428 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7128a6a-4f57-42b8-95b4-dd383e2cddb2 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:05.298834 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7128a6a-4f57-42b8-95b4-dd383e2cddb2 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:05.299114 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7128a6a-4f57-42b8-95b4-dd383e2cddb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:05.320276 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-c7128a6a-4f57-42b8-95b4-dd383e2cddb2 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:05.321532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c7128a6a-4f57-42b8-95b4-dd383e2cddb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:05.392333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1132/2240] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:14:05.403929 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3e6aa-cb3b-4989-871e-687b54766263 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:05.405427 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-eaf3e6aa-cb3b-4989-871e-687b54766263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:05.477175 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3e6aa-cb3b-4989-871e-687b54766263 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:05.481488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9387a5-51d1-4249-9cee-be39331cfcf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:05.482163 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9387a5-51d1-4249-9cee-be39331cfcf9 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:05.482375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9387a5-51d1-4249-9cee-be39331cfcf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:05.482648 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9387a5-51d1-4249-9cee-be39331cfcf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:05.492940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9387a5-51d1-4249-9cee-be39331cfcf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:05.494466 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9387a5-51d1-4249-9cee-be39331cfcf9 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:05.504851 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1109/2241] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:14:05.844450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7128a6a-4f57-42b8-95b4-dd383e2cddb2 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:05.846624 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c7128a6a-4f57-42b8-95b4-dd383e2cddb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:05.864774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7128a6a-4f57-42b8-95b4-dd383e2cddb2 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:05.873395 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ecc13-67b6-4eb8-9671-e4ea8ed0af95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:05.874121 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ecc13-67b6-4eb8-9671-e4ea8ed0af95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:05.874299 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ecc13-67b6-4eb8-9671-e4ea8ed0af95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:05.874622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ecc13-67b6-4eb8-9671-e4ea8ed0af95 admin admin] PATH_INFO: `/v3/projects/5eaa0ee2fa4f46b7829492ec3bd45e00` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:05.875954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ecc13-67b6-4eb8-9671-e4ea8ed0af95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eaa0ee2fa4f46b7829492ec3bd45e00)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:05.877740 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ecc13-67b6-4eb8-9671-e4ea8ed0af95 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:05.935276 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-a35ecc13-67b6-4eb8-9671-e4ea8ed0af95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eaa0ee2fa4f46b7829492ec3bd45e00', 'request_id': 'req-a35ecc13-67b6-4eb8-9671-e4ea8ed0af95'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:05.941422 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-a35ecc13-67b6-4eb8-9671-e4ea8ed0af95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eaa0ee2fa4f46b7829492ec3bd45e00', 'request_id': 'req-a35ecc13-67b6-4eb8-9671-e4ea8ed0af95'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:05.954948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-a35ecc13-67b6-4eb8-9671-e4ea8ed0af95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eaa0ee2fa4f46b7829492ec3bd45e00', 'request_id': 'req-a35ecc13-67b6-4eb8-9671-e4ea8ed0af95'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:05.963540 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:05.972068 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-a35ecc13-67b6-4eb8-9671-e4ea8ed0af95 admin admin] The token cache is being invalidate because project 5eaa0ee2fa4f46b7829492ec3bd45e00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:14:05.972572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-a35ecc13-67b6-4eb8-9671-e4ea8ed0af95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:05.973419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-a35ecc13-67b6-4eb8-9671-e4ea8ed0af95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:05.976191 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1133/2242] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:14:05 2022] DELETE /identity/v3/projects/5eaa0ee2fa4f46b7829492ec3bd45e00 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:05.980392 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9387a5-51d1-4249-9cee-be39331cfcf9 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:05.994452 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9387a5-51d1-4249-9cee-be39331cfcf9 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:06.027191 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:06.067570 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764308cd-1019-41bf-84c2-a5fa4b39c22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:06.068292 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764308cd-1019-41bf-84c2-a5fa4b39c22f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:06.068792 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764308cd-1019-41bf-84c2-a5fa4b39c22f admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:06.069252 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764308cd-1019-41bf-84c2-a5fa4b39c22f admin admin] PATH_INFO: `/v3/users/646183bae905447eb9b3f6ccb948a1a4` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:06.070714 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764308cd-1019-41bf-84c2-a5fa4b39c22f admin admin] RBAC: Authorizing `identity:delete_user(user_id=646183bae905447eb9b3f6ccb948a1a4)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:06.072703 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764308cd-1019-41bf-84c2-a5fa4b39c22f admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:06.125526 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-764308cd-1019-41bf-84c2-a5fa4b39c22f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '646183bae905447eb9b3f6ccb948a1a4'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:06.126205 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-764308cd-1019-41bf-84c2-a5fa4b39c22f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '646183bae905447eb9b3f6ccb948a1a4'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:06.126823 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-764308cd-1019-41bf-84c2-a5fa4b39c22f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '646183bae905447eb9b3f6ccb948a1a4'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:06.136910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-764308cd-1019-41bf-84c2-a5fa4b39c22f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '646183bae905447eb9b3f6ccb948a1a4'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:06.150081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-764308cd-1019-41bf-84c2-a5fa4b39c22f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '646183bae905447eb9b3f6ccb948a1a4'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:06.165387 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1110/2243] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:14:05 2022] DELETE /identity/v3/users/646183bae905447eb9b3f6ccb948a1a4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:06.493472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ecc13-67b6-4eb8-9671-e4ea8ed0af95 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:06.495044 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a35ecc13-67b6-4eb8-9671-e4ea8ed0af95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:06.515105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ecc13-67b6-4eb8-9671-e4ea8ed0af95 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:06.595127 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a6787-cc5f-4dcb-88a8-762a63e2842a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:06.595915 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a6787-cc5f-4dcb-88a8-762a63e2842a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:06.596225 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a6787-cc5f-4dcb-88a8-762a63e2842a admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:06.596503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a6787-cc5f-4dcb-88a8-762a63e2842a admin admin] PATH_INFO: `/v3/projects/bb2975fe3d5449cdaee6e9ef6138f5e4` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:06.597656 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a6787-cc5f-4dcb-88a8-762a63e2842a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb2975fe3d5449cdaee6e9ef6138f5e4)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:06.599705 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a6787-cc5f-4dcb-88a8-762a63e2842a admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:06.663997 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c55a6787-cc5f-4dcb-88a8-762a63e2842a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb2975fe3d5449cdaee6e9ef6138f5e4', 'request_id': 'req-c55a6787-cc5f-4dcb-88a8-762a63e2842a'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:06.671036 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c55a6787-cc5f-4dcb-88a8-762a63e2842a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb2975fe3d5449cdaee6e9ef6138f5e4', 'request_id': 'req-c55a6787-cc5f-4dcb-88a8-762a63e2842a'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:06.683218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c55a6787-cc5f-4dcb-88a8-762a63e2842a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb2975fe3d5449cdaee6e9ef6138f5e4', 'request_id': 'req-c55a6787-cc5f-4dcb-88a8-762a63e2842a'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:06.720728 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c55a6787-cc5f-4dcb-88a8-762a63e2842a admin admin] The token cache is being invalidate because project bb2975fe3d5449cdaee6e9ef6138f5e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:14:06.721181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c55a6787-cc5f-4dcb-88a8-762a63e2842a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:06.722582 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c55a6787-cc5f-4dcb-88a8-762a63e2842a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:06.725809 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1134/2244] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:14:06 2022] DELETE /identity/v3/projects/bb2975fe3d5449cdaee6e9ef6138f5e4 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:06.733703 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752b786-a77f-4a4a-978a-10dec8a219bd None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:06.733993 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752b786-a77f-4a4a-978a-10dec8a219bd None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:06.734272 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752b786-a77f-4a4a-978a-10dec8a219bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:06.755104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-5752b786-a77f-4a4a-978a-10dec8a219bd None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:06.756561 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-5752b786-a77f-4a4a-978a-10dec8a219bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:06.828367 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1111/2245] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:06.837576 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b49453-82a1-4165-b063-6598ad461c68 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:06.837877 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b49453-82a1-4165-b063-6598ad461c68 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:06.838251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b49453-82a1-4165-b063-6598ad461c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:06.868009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-33b49453-82a1-4165-b063-6598ad461c68 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:06.869916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-33b49453-82a1-4165-b063-6598ad461c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:06.878415 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1135/2246] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:06.883641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752b786-a77f-4a4a-978a-10dec8a219bd None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:06.895248 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752b786-a77f-4a4a-978a-10dec8a219bd None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:06.901334 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c13a5-b7e7-45be-9402-3f7549296e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:06.901981 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c13a5-b7e7-45be-9402-3f7549296e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:06.902268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c13a5-b7e7-45be-9402-3f7549296e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:06.902544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c13a5-b7e7-45be-9402-3f7549296e4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:06.903211 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c13a5-b7e7-45be-9402-3f7549296e4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:06.903549 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c13a5-b7e7-45be-9402-3f7549296e4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:06.905220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c13a5-b7e7-45be-9402-3f7549296e4f admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:06.924675 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1112/2247] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:06.926281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b49453-82a1-4165-b063-6598ad461c68 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:06.936423 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b49453-82a1-4165-b063-6598ad461c68 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:06.941403 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3d9b0-715a-444a-b829-bf475fb18593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:06.942005 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3d9b0-715a-444a-b829-bf475fb18593 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:06.942271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3d9b0-715a-444a-b829-bf475fb18593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:06.942546 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3d9b0-715a-444a-b829-bf475fb18593 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:06.943318 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3d9b0-715a-444a-b829-bf475fb18593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:06.945384 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3d9b0-715a-444a-b829-bf475fb18593 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:06.979476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1136/2248] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:14:06 2022] POST /identity/v3/projects => generated 399 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:06.983467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c13a5-b7e7-45be-9402-3f7549296e4f admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:06.997652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c13a5-b7e7-45be-9402-3f7549296e4f admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.033587 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:07.072377 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c42a9-fdaa-4432-9b98-c57f4c1f8936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.073146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c42a9-fdaa-4432-9b98-c57f4c1f8936 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.073498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c42a9-fdaa-4432-9b98-c57f4c1f8936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.073843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c42a9-fdaa-4432-9b98-c57f4c1f8936 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.074567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c42a9-fdaa-4432-9b98-c57f4c1f8936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:07.076645 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c42a9-fdaa-4432-9b98-c57f4c1f8936 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:07.102785 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1113/2249] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:14:06 2022] POST /identity/v3/users => generated 300 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:07.106785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3d9b0-715a-444a-b829-bf475fb18593 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.117687 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3d9b0-715a-444a-b829-bf475fb18593 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.123965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf4be0-55a3-4536-b316-265852175ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.124869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf4be0-55a3-4536-b316-265852175ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.125326 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf4be0-55a3-4536-b316-265852175ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.125600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf4be0-55a3-4536-b316-265852175ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.126361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf4be0-55a3-4536-b316-265852175ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:07.128089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf4be0-55a3-4536-b316-265852175ce6 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:07.142878 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1137/2250] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:07.148568 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c42a9-fdaa-4432-9b98-c57f4c1f8936 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.160378 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c42a9-fdaa-4432-9b98-c57f4c1f8936 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.166433 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61aa5c-0fd7-457c-a80f-eb1028fc0627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.167089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61aa5c-0fd7-457c-a80f-eb1028fc0627 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.167371 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61aa5c-0fd7-457c-a80f-eb1028fc0627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.167671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61aa5c-0fd7-457c-a80f-eb1028fc0627 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.168347 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61aa5c-0fd7-457c-a80f-eb1028fc0627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:07.170035 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61aa5c-0fd7-457c-a80f-eb1028fc0627 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:07.181098 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1114/2251] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:07.186384 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf4be0-55a3-4536-b316-265852175ce6 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.203675 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf4be0-55a3-4536-b316-265852175ce6 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.212890 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f46cd7-b2d1-4140-9f31-75fcb14e4acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.213834 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f46cd7-b2d1-4140-9f31-75fcb14e4acd admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.214226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f46cd7-b2d1-4140-9f31-75fcb14e4acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.214621 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f46cd7-b2d1-4140-9f31-75fcb14e4acd admin admin] PATH_INFO: `/v3/projects/bfa9edfc0c784461aa05f9c9892f84bf/users/1f27fb03fa4f4fa795420ba664136b67/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.239448 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f46cd7-b2d1-4140-9f31-75fcb14e4acd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa9edfc0c784461aa05f9c9892f84bf, user_id=1f27fb03fa4f4fa795420ba664136b67, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:07.245935 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f46cd7-b2d1-4140-9f31-75fcb14e4acd admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:07.267070 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1138/2252] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:14:07 2022] PUT /identity/v3/projects/bfa9edfc0c784461aa05f9c9892f84bf/users/1f27fb03fa4f4fa795420ba664136b67/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:07.283729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68ca46-1e08-4253-b630-c24981ac3026 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.284140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68ca46-1e08-4253-b630-c24981ac3026 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.284592 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68ca46-1e08-4253-b630-c24981ac3026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.322670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-5f68ca46-1e08-4253-b630-c24981ac3026 None None] MFA Rules not processed for user `1f27fb03fa4f4fa795420ba664136b67`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:07.324122 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-5f68ca46-1e08-4253-b630-c24981ac3026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:07.419400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1115/2253] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:07 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:07.424855 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f46cd7-b2d1-4140-9f31-75fcb14e4acd admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.454440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:07.494503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f46cd7-b2d1-4140-9f31-75fcb14e4acd admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.499772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed9a81-f62e-4157-9c2f-46ae97f5577c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.500401 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed9a81-f62e-4157-9c2f-46ae97f5577c admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.500678 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed9a81-f62e-4157-9c2f-46ae97f5577c admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.500952 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed9a81-f62e-4157-9c2f-46ae97f5577c admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.501614 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed9a81-f62e-4157-9c2f-46ae97f5577c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:07.503574 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed9a81-f62e-4157-9c2f-46ae97f5577c admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:07.533105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1139/2254] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:14:07 2022] POST /identity/v3/projects => generated 401 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:07.537246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68ca46-1e08-4253-b630-c24981ac3026 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.549186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68ca46-1e08-4253-b630-c24981ac3026 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.615776 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d77f2-41af-4b32-a88d-e9f5936a5f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.616435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d77f2-41af-4b32-a88d-e9f5936a5f29 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.616838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d77f2-41af-4b32-a88d-e9f5936a5f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.617111 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d77f2-41af-4b32-a88d-e9f5936a5f29 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.617745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d77f2-41af-4b32-a88d-e9f5936a5f29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:07.619882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d77f2-41af-4b32-a88d-e9f5936a5f29 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:07.623581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed9a81-f62e-4157-9c2f-46ae97f5577c admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.625405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8bed9a81-f62e-4157-9c2f-46ae97f5577c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:07.640040 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed9a81-f62e-4157-9c2f-46ae97f5577c admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.645305 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268418b-0bf0-4227-aff0-9df299ee56c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.646045 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268418b-0bf0-4227-aff0-9df299ee56c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.646476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268418b-0bf0-4227-aff0-9df299ee56c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.646848 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268418b-0bf0-4227-aff0-9df299ee56c6 admin admin] PATH_INFO: `/v3/users/958deb6db20945b785b1970682945e7c` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.647767 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1116/2255] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:14:07 2022] POST /identity/v3/users => generated 307 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:07.648833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268418b-0bf0-4227-aff0-9df299ee56c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=958deb6db20945b785b1970682945e7c)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:07.651761 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d77f2-41af-4b32-a88d-e9f5936a5f29 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.656849 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268418b-0bf0-4227-aff0-9df299ee56c6 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:07.664379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d77f2-41af-4b32-a88d-e9f5936a5f29 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.669228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82a880-0916-4d44-9953-5f33b9b27e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.669850 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82a880-0916-4d44-9953-5f33b9b27e88 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.670128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82a880-0916-4d44-9953-5f33b9b27e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.670401 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82a880-0916-4d44-9953-5f33b9b27e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.671057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82a880-0916-4d44-9953-5f33b9b27e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:07.672725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82a880-0916-4d44-9953-5f33b9b27e88 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:07.683746 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1117/2256] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:07.688240 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82a880-0916-4d44-9953-5f33b9b27e88 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.699158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82a880-0916-4d44-9953-5f33b9b27e88 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.703937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1125ede-ad33-400c-9adf-90e4df2dcb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.704707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1125ede-ad33-400c-9adf-90e4df2dcb37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.705001 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1125ede-ad33-400c-9adf-90e4df2dcb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.705341 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1125ede-ad33-400c-9adf-90e4df2dcb37 admin admin] PATH_INFO: `/v3/projects/951ce56f306e4608829a8ba0a96811fa/users/d0e3119d7cee487a9bf61afbdfe8d2fc/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.712662 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d268418b-0bf0-4227-aff0-9df299ee56c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '958deb6db20945b785b1970682945e7c'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:07.714652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d268418b-0bf0-4227-aff0-9df299ee56c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '958deb6db20945b785b1970682945e7c'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:07.715547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d268418b-0bf0-4227-aff0-9df299ee56c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '958deb6db20945b785b1970682945e7c'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:07.726346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d268418b-0bf0-4227-aff0-9df299ee56c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '958deb6db20945b785b1970682945e7c'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:07.728572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1125ede-ad33-400c-9adf-90e4df2dcb37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=951ce56f306e4608829a8ba0a96811fa, user_id=d0e3119d7cee487a9bf61afbdfe8d2fc, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:07.731943 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1125ede-ad33-400c-9adf-90e4df2dcb37 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:07.735825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:07.741260 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d268418b-0bf0-4227-aff0-9df299ee56c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '958deb6db20945b785b1970682945e7c'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:07.756457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1118/2257] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:14:07 2022] PUT /identity/v3/projects/951ce56f306e4608829a8ba0a96811fa/users/d0e3119d7cee487a9bf61afbdfe8d2fc/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:07.757379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1125ede-ad33-400c-9adf-90e4df2dcb37 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.760344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1140/2258] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:14:07 2022] DELETE /identity/v3/users/958deb6db20945b785b1970682945e7c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:07.760609 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d1125ede-ad33-400c-9adf-90e4df2dcb37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:07.764944 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8cb73-13e7-4192-98fd-ff09a9bcc59d None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.765289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8cb73-13e7-4192-98fd-ff09a9bcc59d None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.768507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8cb73-13e7-4192-98fd-ff09a9bcc59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.808776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-22f8cb73-13e7-4192-98fd-ff09a9bcc59d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:07.808776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-22f8cb73-13e7-4192-98fd-ff09a9bcc59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:07.928247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1141/2259] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:07.937740 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1125ede-ad33-400c-9adf-90e4df2dcb37 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:07.939816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8cb73-13e7-4192-98fd-ff09a9bcc59d None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:07.941526 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-22f8cb73-13e7-4192-98fd-ff09a9bcc59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:07.964286 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d93a2-6f23-498a-a4e0-8ef48976e496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:07.965028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d93a2-6f23-498a-a4e0-8ef48976e496 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:07.965373 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d93a2-6f23-498a-a4e0-8ef48976e496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:07.965814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d93a2-6f23-498a-a4e0-8ef48976e496 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:07.966539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d93a2-6f23-498a-a4e0-8ef48976e496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:07.971336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d93a2-6f23-498a-a4e0-8ef48976e496 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:07.984870 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1119/2260] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:07.995363 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:07.997514 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d93a2-6f23-498a-a4e0-8ef48976e496 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:08.017728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d93a2-6f23-498a-a4e0-8ef48976e496 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:08.027247 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08dbb6c-8bab-4f5e-adb7-e216ffe3f8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:08.028099 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dbb6c-8bab-4f5e-adb7-e216ffe3f8aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:08.028427 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dbb6c-8bab-4f5e-adb7-e216ffe3f8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:08.028775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dbb6c-8bab-4f5e-adb7-e216ffe3f8aa admin admin] PATH_INFO: `/v3/projects/951ce56f306e4608829a8ba0a96811fa/users/d0e3119d7cee487a9bf61afbdfe8d2fc/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:08.030821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08dbb6c-8bab-4f5e-adb7-e216ffe3f8aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=951ce56f306e4608829a8ba0a96811fa, user_id=d0e3119d7cee487a9bf61afbdfe8d2fc, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:08.035759 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08dbb6c-8bab-4f5e-adb7-e216ffe3f8aa admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:08.055818 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1120/2261] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:14:07 2022] PUT /identity/v3/projects/951ce56f306e4608829a8ba0a96811fa/users/d0e3119d7cee487a9bf61afbdfe8d2fc/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:08.066729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906a333-d80d-4174-93f5-ff19caea0062 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:08.067245 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906a333-d80d-4174-93f5-ff19caea0062 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:08.067606 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906a333-d80d-4174-93f5-ff19caea0062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:08.093122 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8cb73-13e7-4192-98fd-ff09a9bcc59d None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:08.116271 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-c906a333-d80d-4174-93f5-ff19caea0062 None None] MFA Rules not processed for user `d0e3119d7cee487a9bf61afbdfe8d2fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:08.117885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c906a333-d80d-4174-93f5-ff19caea0062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:08.193813 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d01572-dae2-4f8b-9733-63a1541c55ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:08.194414 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d01572-dae2-4f8b-9733-63a1541c55ba service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:08.194691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d01572-dae2-4f8b-9733-63a1541c55ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:08.194942 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d01572-dae2-4f8b-9733-63a1541c55ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:08.233069 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1121/2262] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:08 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:14:08.246184 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906a333-d80d-4174-93f5-ff19caea0062 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:08.248676 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c906a333-d80d-4174-93f5-ff19caea0062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:08.291113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d01572-dae2-4f8b-9733-63a1541c55ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:08.293113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d01572-dae2-4f8b-9733-63a1541c55ba service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:08.311244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1142/2263] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:14:08.345561 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906a333-d80d-4174-93f5-ff19caea0062 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:08.350563 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98af8fa-eb9a-45f5-bdee-70f4823d659c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:08.351220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98af8fa-eb9a-45f5-bdee-70f4823d659c service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:08.352066 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98af8fa-eb9a-45f5-bdee-70f4823d659c service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:08.352066 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98af8fa-eb9a-45f5-bdee-70f4823d659c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:08.362564 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98af8fa-eb9a-45f5-bdee-70f4823d659c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:08.364305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98af8fa-eb9a-45f5-bdee-70f4823d659c service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:08.377546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1122/2264] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:08 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:14:08.480439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d01572-dae2-4f8b-9733-63a1541c55ba service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:08.496989 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d01572-dae2-4f8b-9733-63a1541c55ba service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:08.503996 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d49f6d-af49-4a7c-bd81-135d996963c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:08.504722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d49f6d-af49-4a7c-bd81-135d996963c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:08.505095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d49f6d-af49-4a7c-bd81-135d996963c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:08.505657 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d49f6d-af49-4a7c-bd81-135d996963c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:08.508326 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d3d49f6d-af49-4a7c-bd81-135d996963c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:08.590344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d49f6d-af49-4a7c-bd81-135d996963c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:08.592601 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d49f6d-af49-4a7c-bd81-135d996963c6 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:08.605382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1143/2265] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:08 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:14:08.658786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98af8fa-eb9a-45f5-bdee-70f4823d659c service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:08.660383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f98af8fa-eb9a-45f5-bdee-70f4823d659c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:08.697989 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98af8fa-eb9a-45f5-bdee-70f4823d659c service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:08.703441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c49cfc-f2ef-47fe-8a68-ad86e9c370c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:08.704098 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c49cfc-f2ef-47fe-8a68-ad86e9c370c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:08.704365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c49cfc-f2ef-47fe-8a68-ad86e9c370c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:08.704637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c49cfc-f2ef-47fe-8a68-ad86e9c370c1 admin admin] PATH_INFO: `/v3/projects/7cf48cd3747b48c3acaaae55885222bd` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:08.705759 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c49cfc-f2ef-47fe-8a68-ad86e9c370c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cf48cd3747b48c3acaaae55885222bd)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:08.708110 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c49cfc-f2ef-47fe-8a68-ad86e9c370c1 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:08.709971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:08.763500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-16c49cfc-f2ef-47fe-8a68-ad86e9c370c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cf48cd3747b48c3acaaae55885222bd', 'request_id': 'req-16c49cfc-f2ef-47fe-8a68-ad86e9c370c1'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:08.769575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-16c49cfc-f2ef-47fe-8a68-ad86e9c370c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cf48cd3747b48c3acaaae55885222bd', 'request_id': 'req-16c49cfc-f2ef-47fe-8a68-ad86e9c370c1'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:08.782752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-16c49cfc-f2ef-47fe-8a68-ad86e9c370c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cf48cd3747b48c3acaaae55885222bd', 'request_id': 'req-16c49cfc-f2ef-47fe-8a68-ad86e9c370c1'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:08.791396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-16c49cfc-f2ef-47fe-8a68-ad86e9c370c1 admin admin] The token cache is being invalidate because project 7cf48cd3747b48c3acaaae55885222bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:14:08.791750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-16c49cfc-f2ef-47fe-8a68-ad86e9c370c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:08.792619 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-16c49cfc-f2ef-47fe-8a68-ad86e9c370c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:08.794905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1123/2266] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:14:08 2022] DELETE /identity/v3/projects/7cf48cd3747b48c3acaaae55885222bd => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:08.799776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d49f6d-af49-4a7c-bd81-135d996963c6 service cinder] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:08.813438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d49f6d-af49-4a7c-bd81-135d996963c6 service cinder] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:08.844013 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:08.880254 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdb020-035b-42dc-a2d5-413331623c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:08.880785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdb020-035b-42dc-a2d5-413331623c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:08.881068 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdb020-035b-42dc-a2d5-413331623c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:08.881348 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdb020-035b-42dc-a2d5-413331623c5b admin admin] PATH_INFO: `/v3/users/4f3905476438460da404d8c86c8e75ba` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:08.882610 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdb020-035b-42dc-a2d5-413331623c5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f3905476438460da404d8c86c8e75ba)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:08.885268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdb020-035b-42dc-a2d5-413331623c5b admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:08.988366 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-87bdb020-035b-42dc-a2d5-413331623c5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f3905476438460da404d8c86c8e75ba'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:08.989046 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-87bdb020-035b-42dc-a2d5-413331623c5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f3905476438460da404d8c86c8e75ba'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:08.990656 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-87bdb020-035b-42dc-a2d5-413331623c5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f3905476438460da404d8c86c8e75ba'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:09.001720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-87bdb020-035b-42dc-a2d5-413331623c5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f3905476438460da404d8c86c8e75ba'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:09.065848 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-87bdb020-035b-42dc-a2d5-413331623c5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f3905476438460da404d8c86c8e75ba'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:09.085072 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1144/2267] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:14:08 2022] DELETE /identity/v3/users/4f3905476438460da404d8c86c8e75ba => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:09.415811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c49cfc-f2ef-47fe-8a68-ad86e9c370c1 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:09.418347 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-16c49cfc-f2ef-47fe-8a68-ad86e9c370c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:09.440915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c49cfc-f2ef-47fe-8a68-ad86e9c370c1 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:09.525751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf0ca9-8a11-49f9-81ec-1d736e802fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:09.526267 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf0ca9-8a11-49f9-81ec-1d736e802fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:09.526539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf0ca9-8a11-49f9-81ec-1d736e802fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:09.526775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf0ca9-8a11-49f9-81ec-1d736e802fa7 admin admin] PATH_INFO: `/v3/projects/c8bc3cbe55f041d6a39e0583134170c5` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:09.528565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf0ca9-8a11-49f9-81ec-1d736e802fa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8bc3cbe55f041d6a39e0583134170c5)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:09.530782 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf0ca9-8a11-49f9-81ec-1d736e802fa7 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:09.625370 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-0fbf0ca9-8a11-49f9-81ec-1d736e802fa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8bc3cbe55f041d6a39e0583134170c5', 'request_id': 'req-0fbf0ca9-8a11-49f9-81ec-1d736e802fa7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:09.650291 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-0fbf0ca9-8a11-49f9-81ec-1d736e802fa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8bc3cbe55f041d6a39e0583134170c5', 'request_id': 'req-0fbf0ca9-8a11-49f9-81ec-1d736e802fa7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:09.671206 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-0fbf0ca9-8a11-49f9-81ec-1d736e802fa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8bc3cbe55f041d6a39e0583134170c5', 'request_id': 'req-0fbf0ca9-8a11-49f9-81ec-1d736e802fa7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:09.704865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-0fbf0ca9-8a11-49f9-81ec-1d736e802fa7 admin admin] The token cache is being invalidate because project c8bc3cbe55f041d6a39e0583134170c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:14:09.705250 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-0fbf0ca9-8a11-49f9-81ec-1d736e802fa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:09.706067 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-0fbf0ca9-8a11-49f9-81ec-1d736e802fa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:09.709478 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1124/2268] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:14:09 2022] DELETE /identity/v3/projects/c8bc3cbe55f041d6a39e0583134170c5 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:09.720843 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdb020-035b-42dc-a2d5-413331623c5b admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:09.722660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-87bdb020-035b-42dc-a2d5-413331623c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:09.738936 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdb020-035b-42dc-a2d5-413331623c5b admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:09.820474 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c52c6-97cd-4c5f-a542-f55e8f7e8aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:09.821236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c52c6-97cd-4c5f-a542-f55e8f7e8aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:09.821778 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c52c6-97cd-4c5f-a542-f55e8f7e8aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:09.822326 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c52c6-97cd-4c5f-a542-f55e8f7e8aee admin admin] PATH_INFO: `/v3/users/cbfad94c7dad4a9184aa30d81876abd1` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:09.824708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c52c6-97cd-4c5f-a542-f55e8f7e8aee admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbfad94c7dad4a9184aa30d81876abd1)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:09.827813 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c52c6-97cd-4c5f-a542-f55e8f7e8aee admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:09.897523 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e42c52c6-97cd-4c5f-a542-f55e8f7e8aee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbfad94c7dad4a9184aa30d81876abd1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:09.898345 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e42c52c6-97cd-4c5f-a542-f55e8f7e8aee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbfad94c7dad4a9184aa30d81876abd1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:09.899212 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e42c52c6-97cd-4c5f-a542-f55e8f7e8aee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbfad94c7dad4a9184aa30d81876abd1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:09.903848 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:09.913321 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e42c52c6-97cd-4c5f-a542-f55e8f7e8aee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbfad94c7dad4a9184aa30d81876abd1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:09.926633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e42c52c6-97cd-4c5f-a542-f55e8f7e8aee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbfad94c7dad4a9184aa30d81876abd1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:09.942552 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1145/2269] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:14:09 2022] DELETE /identity/v3/users/cbfad94c7dad4a9184aa30d81876abd1 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:10.294156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf0ca9-8a11-49f9-81ec-1d736e802fa7 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:10.295765 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-0fbf0ca9-8a11-49f9-81ec-1d736e802fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:10.311763 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf0ca9-8a11-49f9-81ec-1d736e802fa7 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:10.348721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:10.396525 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5c8cf-6c09-4d43-b87f-fdc520160973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:10.397354 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5c8cf-6c09-4d43-b87f-fdc520160973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:10.397694 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5c8cf-6c09-4d43-b87f-fdc520160973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:10.398040 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5c8cf-6c09-4d43-b87f-fdc520160973 admin admin] PATH_INFO: `/v3/projects/37fbb504c8464bc0871f6b9cbbe6c545` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:10.399246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff5c8cf-6c09-4d43-b87f-fdc520160973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37fbb504c8464bc0871f6b9cbbe6c545)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:10.401315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff5c8cf-6c09-4d43-b87f-fdc520160973 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:10.453220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-7ff5c8cf-6c09-4d43-b87f-fdc520160973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37fbb504c8464bc0871f6b9cbbe6c545', 'request_id': 'req-7ff5c8cf-6c09-4d43-b87f-fdc520160973'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:10.459500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-7ff5c8cf-6c09-4d43-b87f-fdc520160973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37fbb504c8464bc0871f6b9cbbe6c545', 'request_id': 'req-7ff5c8cf-6c09-4d43-b87f-fdc520160973'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:10.471673 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-7ff5c8cf-6c09-4d43-b87f-fdc520160973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37fbb504c8464bc0871f6b9cbbe6c545', 'request_id': 'req-7ff5c8cf-6c09-4d43-b87f-fdc520160973'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:10.485396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-7ff5c8cf-6c09-4d43-b87f-fdc520160973 admin admin] The token cache is being invalidate because project 37fbb504c8464bc0871f6b9cbbe6c545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:14:10.485908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-7ff5c8cf-6c09-4d43-b87f-fdc520160973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:10.486796 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-7ff5c8cf-6c09-4d43-b87f-fdc520160973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:10.488745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1125/2270] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:14:10 2022] DELETE /identity/v3/projects/37fbb504c8464bc0871f6b9cbbe6c545 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:10.494392 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e0b77-2598-4673-91c6-0d966ee921da None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:10.495155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e0b77-2598-4673-91c6-0d966ee921da None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:10.495155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e0b77-2598-4673-91c6-0d966ee921da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:10.516865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-7e5e0b77-2598-4673-91c6-0d966ee921da None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:10.518083 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-7e5e0b77-2598-4673-91c6-0d966ee921da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:10.586994 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1146/2271] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:10.595282 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2e542-e543-4b04-864e-a5437d8e3713 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:10.595615 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2e542-e543-4b04-864e-a5437d8e3713 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:10.595910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2e542-e543-4b04-864e-a5437d8e3713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:10.617332 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-4fb2e542-e543-4b04-864e-a5437d8e3713 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:10.618564 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-4fb2e542-e543-4b04-864e-a5437d8e3713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:10.626982 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1126/2272] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:10.631914 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e0b77-2598-4673-91c6-0d966ee921da None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:10.642221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e0b77-2598-4673-91c6-0d966ee921da None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:10.647723 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba8400-d8c8-4505-8a17-20b4cf73493c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:10.648335 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba8400-d8c8-4505-8a17-20b4cf73493c admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:10.648609 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba8400-d8c8-4505-8a17-20b4cf73493c admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:10.648877 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba8400-d8c8-4505-8a17-20b4cf73493c admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:10.649542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba8400-d8c8-4505-8a17-20b4cf73493c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:10.649878 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba8400-d8c8-4505-8a17-20b4cf73493c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:10.651549 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba8400-d8c8-4505-8a17-20b4cf73493c admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:10.668835 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1147/2273] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:10.670842 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2e542-e543-4b04-864e-a5437d8e3713 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:10.683347 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2e542-e543-4b04-864e-a5437d8e3713 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:10.688675 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96311dff-7340-4ce3-a947-f9af8a48275e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:10.689298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96311dff-7340-4ce3-a947-f9af8a48275e admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:10.689576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96311dff-7340-4ce3-a947-f9af8a48275e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:10.689860 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96311dff-7340-4ce3-a947-f9af8a48275e admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:10.690489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96311dff-7340-4ce3-a947-f9af8a48275e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:10.692524 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96311dff-7340-4ce3-a947-f9af8a48275e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:10.728481 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1127/2274] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:14:10 2022] POST /identity/v3/projects => generated 369 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:10.732439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba8400-d8c8-4505-8a17-20b4cf73493c admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:10.742799 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba8400-d8c8-4505-8a17-20b4cf73493c admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:10.803649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e4a78-6739-4d9e-8016-0917aa89a55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:10.804441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e4a78-6739-4d9e-8016-0917aa89a55b admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:10.805161 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e4a78-6739-4d9e-8016-0917aa89a55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:10.805161 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e4a78-6739-4d9e-8016-0917aa89a55b admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:10.805603 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e4a78-6739-4d9e-8016-0917aa89a55b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:10.807565 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e4a78-6739-4d9e-8016-0917aa89a55b admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:10.833561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1148/2275] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:14:10 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:10.839771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96311dff-7340-4ce3-a947-f9af8a48275e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:10.852808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96311dff-7340-4ce3-a947-f9af8a48275e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:10.857640 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05455424-9891-4bed-a3e6-832caba4b68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:10.858262 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05455424-9891-4bed-a3e6-832caba4b68b admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:10.858534 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05455424-9891-4bed-a3e6-832caba4b68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:10.858800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05455424-9891-4bed-a3e6-832caba4b68b admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:10.859478 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05455424-9891-4bed-a3e6-832caba4b68b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:10.861134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05455424-9891-4bed-a3e6-832caba4b68b admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:10.871904 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1128/2276] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:14:10.875974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e4a78-6739-4d9e-8016-0917aa89a55b admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:10.886571 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e4a78-6739-4d9e-8016-0917aa89a55b admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:10.892155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d45201-d652-4132-82a4-83d58a0e76eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:10.892781 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d45201-d652-4132-82a4-83d58a0e76eb admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:10.893681 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d45201-d652-4132-82a4-83d58a0e76eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:10.893953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d45201-d652-4132-82a4-83d58a0e76eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:10.894603 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d45201-d652-4132-82a4-83d58a0e76eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:10.896273 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d45201-d652-4132-82a4-83d58a0e76eb admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:10.908047 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1149/2277] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:10.912321 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05455424-9891-4bed-a3e6-832caba4b68b admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:10.923947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05455424-9891-4bed-a3e6-832caba4b68b admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:10.929774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c2208-9b3a-4842-adde-43d5a41e1e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:10.930440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c2208-9b3a-4842-adde-43d5a41e1e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:10.930716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c2208-9b3a-4842-adde-43d5a41e1e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:10.930986 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c2208-9b3a-4842-adde-43d5a41e1e09 admin admin] PATH_INFO: `/v3/projects/711ca60ec25e46e3acd135dac97476f5/users/b016f4b3e0ce4197a63d139e72c1856b/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:10.947062 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c2208-9b3a-4842-adde-43d5a41e1e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=711ca60ec25e46e3acd135dac97476f5, user_id=b016f4b3e0ce4197a63d139e72c1856b, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:10.950181 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c2208-9b3a-4842-adde-43d5a41e1e09 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:10.965827 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1129/2278] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:14:10 2022] PUT /identity/v3/projects/711ca60ec25e46e3acd135dac97476f5/users/b016f4b3e0ce4197a63d139e72c1856b/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:10.976823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c38f1-6d1e-4132-9a4c-a1689f80a3ad None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:10.977107 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c38f1-6d1e-4132-9a4c-a1689f80a3ad None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:10.977380 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c38f1-6d1e-4132-9a4c-a1689f80a3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:11.010419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-396c38f1-6d1e-4132-9a4c-a1689f80a3ad None None] MFA Rules not processed for user `b016f4b3e0ce4197a63d139e72c1856b`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:11.011779 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-396c38f1-6d1e-4132-9a4c-a1689f80a3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:11.032248 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:11.081112 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1150/2279] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:14:11.093217 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c2208-9b3a-4842-adde-43d5a41e1e09 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:11.094865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-702c2208-9b3a-4842-adde-43d5a41e1e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:11.150802 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:11.156686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c2208-9b3a-4842-adde-43d5a41e1e09 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:11.161462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18e378-d208-449d-b03f-d3acbc045c41 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57a5f04cb9674b86a6b4b9a472e1afc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:11.162280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18e378-d208-449d-b03f-d3acbc045c41 service glance] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:11.162645 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18e378-d208-449d-b03f-d3acbc045c41 service glance] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:11.162918 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18e378-d208-449d-b03f-d3acbc045c41 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:11.171640 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18e378-d208-449d-b03f-d3acbc045c41 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:11.173181 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18e378-d208-449d-b03f-d3acbc045c41 service glance] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:11.182668 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1130/2280] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:14:13.178547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c38f1-6d1e-4132-9a4c-a1689f80a3ad None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:13.183608 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-396c38f1-6d1e-4132-9a4c-a1689f80a3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:13.303933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c38f1-6d1e-4132-9a4c-a1689f80a3ad None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:13.309079 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18dcd7-12ce-4878-a6b3-4bdfcd899f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:13.309807 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18dcd7-12ce-4878-a6b3-4bdfcd899f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:13.310088 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18dcd7-12ce-4878-a6b3-4bdfcd899f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:13.310387 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18dcd7-12ce-4878-a6b3-4bdfcd899f22 admin admin] PATH_INFO: `/v3/users/1f27fb03fa4f4fa795420ba664136b67` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:13.311638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18dcd7-12ce-4878-a6b3-4bdfcd899f22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f27fb03fa4f4fa795420ba664136b67)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:13.313631 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18dcd7-12ce-4878-a6b3-4bdfcd899f22 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:13.379388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1c18dcd7-12ce-4878-a6b3-4bdfcd899f22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f27fb03fa4f4fa795420ba664136b67'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:13.380033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1c18dcd7-12ce-4878-a6b3-4bdfcd899f22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f27fb03fa4f4fa795420ba664136b67'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:13.380833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1c18dcd7-12ce-4878-a6b3-4bdfcd899f22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f27fb03fa4f4fa795420ba664136b67'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:13.389165 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1c18dcd7-12ce-4878-a6b3-4bdfcd899f22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f27fb03fa4f4fa795420ba664136b67'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:13.402829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1c18dcd7-12ce-4878-a6b3-4bdfcd899f22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f27fb03fa4f4fa795420ba664136b67'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:13.418537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1151/2281] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:14:13 2022] DELETE /identity/v3/users/1f27fb03fa4f4fa795420ba664136b67 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:13.423736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1e683-3fc6-4814-a039-1b030a31db38 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:13.424285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1e683-3fc6-4814-a039-1b030a31db38 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:13.424719 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1e683-3fc6-4814-a039-1b030a31db38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:13.446214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-4ff1e683-3fc6-4814-a039-1b030a31db38 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:13.448366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-4ff1e683-3fc6-4814-a039-1b030a31db38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:13.522129 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1131/2282] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:13.533135 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18dcd7-12ce-4878-a6b3-4bdfcd899f22 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:13.534603 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-1c18dcd7-12ce-4878-a6b3-4bdfcd899f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:13.581445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:13.600278 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18dcd7-12ce-4878-a6b3-4bdfcd899f22 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:13.609527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d2b6d-f5e0-48b3-8eee-10b21842a64f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:13.610410 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d2b6d-f5e0-48b3-8eee-10b21842a64f service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:13.610727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d2b6d-f5e0-48b3-8eee-10b21842a64f service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:13.611155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d2b6d-f5e0-48b3-8eee-10b21842a64f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:13.623827 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d2b6d-f5e0-48b3-8eee-10b21842a64f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:13.625627 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d2b6d-f5e0-48b3-8eee-10b21842a64f service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:13.636359 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1152/2283] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:14:13.793219 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9c87f-c4cd-4c02-9745-3381d4fca289 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:13.793667 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9c87f-c4cd-4c02-9745-3381d4fca289 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:13.794033 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9c87f-c4cd-4c02-9745-3381d4fca289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:13.827194 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-4fa9c87f-c4cd-4c02-9745-3381d4fca289 None None] MFA Rules not processed for user `9f4d65ffaab942f29d449b64805c765a`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:13.828480 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-4fa9c87f-c4cd-4c02-9745-3381d4fca289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:13.914907 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1132/2284] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:14:13.968025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d2b6d-f5e0-48b3-8eee-10b21842a64f service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:13.969934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6d4d2b6d-f5e0-48b3-8eee-10b21842a64f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:13.985062 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d2b6d-f5e0-48b3-8eee-10b21842a64f service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:13.991792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1fd0b-58bc-41ad-9f9a-dd4199e26d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:13.993175 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1fd0b-58bc-41ad-9f9a-dd4199e26d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:13.993597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1fd0b-58bc-41ad-9f9a-dd4199e26d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:13.994019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1fd0b-58bc-41ad-9f9a-dd4199e26d75 admin admin] PATH_INFO: `/v3/projects/bfa9edfc0c784461aa05f9c9892f84bf` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:13.995321 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1fd0b-58bc-41ad-9f9a-dd4199e26d75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfa9edfc0c784461aa05f9c9892f84bf)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:13.998024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1fd0b-58bc-41ad-9f9a-dd4199e26d75 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:14.048193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-14e1fd0b-58bc-41ad-9f9a-dd4199e26d75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfa9edfc0c784461aa05f9c9892f84bf', 'request_id': 'req-14e1fd0b-58bc-41ad-9f9a-dd4199e26d75'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:14.057568 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-14e1fd0b-58bc-41ad-9f9a-dd4199e26d75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfa9edfc0c784461aa05f9c9892f84bf', 'request_id': 'req-14e1fd0b-58bc-41ad-9f9a-dd4199e26d75'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:14.065186 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-14e1fd0b-58bc-41ad-9f9a-dd4199e26d75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfa9edfc0c784461aa05f9c9892f84bf', 'request_id': 'req-14e1fd0b-58bc-41ad-9f9a-dd4199e26d75'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:14.076805 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-14e1fd0b-58bc-41ad-9f9a-dd4199e26d75 admin admin] The token cache is being invalidate because project bfa9edfc0c784461aa05f9c9892f84bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:14:14.077091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-14e1fd0b-58bc-41ad-9f9a-dd4199e26d75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:14.077733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-14e1fd0b-58bc-41ad-9f9a-dd4199e26d75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:14.079242 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1153/2285] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:14:13 2022] DELETE /identity/v3/projects/bfa9edfc0c784461aa05f9c9892f84bf => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:14.083681 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9c87f-c4cd-4c02-9745-3381d4fca289 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:14.094847 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9c87f-c4cd-4c02-9745-3381d4fca289 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:14.159805 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf290627-b341-4383-91a2-5a801f63b5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:14.160576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf290627-b341-4383-91a2-5a801f63b5b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:14.160720 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf290627-b341-4383-91a2-5a801f63b5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:14.160891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf290627-b341-4383-91a2-5a801f63b5b1 admin admin] PATH_INFO: `/v3/users/d0e3119d7cee487a9bf61afbdfe8d2fc` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:14.162058 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf290627-b341-4383-91a2-5a801f63b5b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0e3119d7cee487a9bf61afbdfe8d2fc)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:14.164111 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf290627-b341-4383-91a2-5a801f63b5b1 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:14.209176 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:14.219021 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-bf290627-b341-4383-91a2-5a801f63b5b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0e3119d7cee487a9bf61afbdfe8d2fc'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:14.219746 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-bf290627-b341-4383-91a2-5a801f63b5b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0e3119d7cee487a9bf61afbdfe8d2fc'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:14.220381 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-bf290627-b341-4383-91a2-5a801f63b5b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0e3119d7cee487a9bf61afbdfe8d2fc'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:14.228597 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-bf290627-b341-4383-91a2-5a801f63b5b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0e3119d7cee487a9bf61afbdfe8d2fc'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:14.242233 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-bf290627-b341-4383-91a2-5a801f63b5b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0e3119d7cee487a9bf61afbdfe8d2fc'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:14.258454 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1133/2286] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:14:14 2022] DELETE /identity/v3/users/d0e3119d7cee487a9bf61afbdfe8d2fc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:14.548298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1fd0b-58bc-41ad-9f9a-dd4199e26d75 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:14.550319 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-14e1fd0b-58bc-41ad-9f9a-dd4199e26d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:14.565906 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1fd0b-58bc-41ad-9f9a-dd4199e26d75 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:14.636677 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5f7ef-8e2e-44ef-b382-a59eda86d9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:14.637335 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5f7ef-8e2e-44ef-b382-a59eda86d9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:14.637614 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5f7ef-8e2e-44ef-b382-a59eda86d9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:14.637888 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5f7ef-8e2e-44ef-b382-a59eda86d9d4 admin admin] PATH_INFO: `/v3/projects/951ce56f306e4608829a8ba0a96811fa` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:14.640178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5f7ef-8e2e-44ef-b382-a59eda86d9d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=951ce56f306e4608829a8ba0a96811fa)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:14.642765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5f7ef-8e2e-44ef-b382-a59eda86d9d4 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:14.732886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-cae5f7ef-8e2e-44ef-b382-a59eda86d9d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '951ce56f306e4608829a8ba0a96811fa', 'request_id': 'req-cae5f7ef-8e2e-44ef-b382-a59eda86d9d4'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:14.739297 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-cae5f7ef-8e2e-44ef-b382-a59eda86d9d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '951ce56f306e4608829a8ba0a96811fa', 'request_id': 'req-cae5f7ef-8e2e-44ef-b382-a59eda86d9d4'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:14.749537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-cae5f7ef-8e2e-44ef-b382-a59eda86d9d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '951ce56f306e4608829a8ba0a96811fa', 'request_id': 'req-cae5f7ef-8e2e-44ef-b382-a59eda86d9d4'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:14.752055 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:14.763888 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-cae5f7ef-8e2e-44ef-b382-a59eda86d9d4 admin admin] The token cache is being invalidate because project 951ce56f306e4608829a8ba0a96811fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:14:14.764189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-cae5f7ef-8e2e-44ef-b382-a59eda86d9d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:14.765017 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-cae5f7ef-8e2e-44ef-b382-a59eda86d9d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:14.766792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1154/2287] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:14:14 2022] DELETE /identity/v3/projects/951ce56f306e4608829a8ba0a96811fa => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:14.774892 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea4532-9579-4fb8-bda4-466eef27b489 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:14.775385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea4532-9579-4fb8-bda4-466eef27b489 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:14.775505 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea4532-9579-4fb8-bda4-466eef27b489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:14.796689 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-c5ea4532-9579-4fb8-bda4-466eef27b489 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:14.797945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c5ea4532-9579-4fb8-bda4-466eef27b489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:14.862146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1134/2288] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:14:14.870763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299e59e-5610-4300-b665-a3f2769664b6 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:14.871073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299e59e-5610-4300-b665-a3f2769664b6 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:14.871314 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299e59e-5610-4300-b665-a3f2769664b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:14.894162 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-9299e59e-5610-4300-b665-a3f2769664b6 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:14.895539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9299e59e-5610-4300-b665-a3f2769664b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:14.905420 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1155/2289] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:14:14.909839 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea4532-9579-4fb8-bda4-466eef27b489 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:14.920657 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea4532-9579-4fb8-bda4-466eef27b489 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:14.925296 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdfad8-41a2-477f-b008-1c4cf31c5b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:14.925913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdfad8-41a2-477f-b008-1c4cf31c5b84 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:14.926187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdfad8-41a2-477f-b008-1c4cf31c5b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:14.926460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdfad8-41a2-477f-b008-1c4cf31c5b84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:14.927117 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cdfad8-41a2-477f-b008-1c4cf31c5b84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:14.927431 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cdfad8-41a2-477f-b008-1c4cf31c5b84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:14.929720 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cdfad8-41a2-477f-b008-1c4cf31c5b84 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:14.941990 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1135/2290] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:14.949576 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299e59e-5610-4300-b665-a3f2769664b6 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:14.958530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299e59e-5610-4300-b665-a3f2769664b6 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:14.964458 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbba730-e321-4f0c-8889-018ef9aadbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:14.965086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbba730-e321-4f0c-8889-018ef9aadbbc admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:14.965366 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbba730-e321-4f0c-8889-018ef9aadbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:14.965641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbba730-e321-4f0c-8889-018ef9aadbbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:14.966297 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbba730-e321-4f0c-8889-018ef9aadbbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:14.969464 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbba730-e321-4f0c-8889-018ef9aadbbc admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:15.007204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1156/2291] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:14:14 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:15.011897 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdfad8-41a2-477f-b008-1c4cf31c5b84 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:15.023351 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdfad8-41a2-477f-b008-1c4cf31c5b84 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:15.089973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476bfbf-b997-424d-a060-305571f97fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:15.090598 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476bfbf-b997-424d-a060-305571f97fee admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:15.090939 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476bfbf-b997-424d-a060-305571f97fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:15.091221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476bfbf-b997-424d-a060-305571f97fee admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:15.091987 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476bfbf-b997-424d-a060-305571f97fee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:15.093902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476bfbf-b997-424d-a060-305571f97fee admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:15.119821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1136/2292] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:14:15 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:15.124105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbba730-e321-4f0c-8889-018ef9aadbbc admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:15.136263 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbba730-e321-4f0c-8889-018ef9aadbbc admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:15.141684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae0acf-877b-4ecb-9ed8-5a6f5b50ca54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:15.142222 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae0acf-877b-4ecb-9ed8-5a6f5b50ca54 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:15.142641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae0acf-877b-4ecb-9ed8-5a6f5b50ca54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:15.142935 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae0acf-877b-4ecb-9ed8-5a6f5b50ca54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:15.143631 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae0acf-877b-4ecb-9ed8-5a6f5b50ca54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:15.145285 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae0acf-877b-4ecb-9ed8-5a6f5b50ca54 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:15.156215 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1157/2293] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:15.160672 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476bfbf-b997-424d-a060-305571f97fee admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:15.172113 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476bfbf-b997-424d-a060-305571f97fee admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:15.178850 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03cfc2c-3947-4a78-932e-8ab624cd6896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:15.179527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cfc2c-3947-4a78-932e-8ab624cd6896 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:15.179810 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cfc2c-3947-4a78-932e-8ab624cd6896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:15.180082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cfc2c-3947-4a78-932e-8ab624cd6896 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:15.180735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03cfc2c-3947-4a78-932e-8ab624cd6896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:15.182418 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03cfc2c-3947-4a78-932e-8ab624cd6896 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:15.195692 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1137/2294] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:15.202497 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae0acf-877b-4ecb-9ed8-5a6f5b50ca54 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:15.217016 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae0acf-877b-4ecb-9ed8-5a6f5b50ca54 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:15.222428 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea372f4d-1c07-4ff3-85ec-e629a11945cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:15.223158 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea372f4d-1c07-4ff3-85ec-e629a11945cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:15.223466 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea372f4d-1c07-4ff3-85ec-e629a11945cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:15.223758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea372f4d-1c07-4ff3-85ec-e629a11945cd admin admin] PATH_INFO: `/v3/projects/3d91979a6ce9468c990b9e08e267d329/users/6ed299381d5b49fb8577287af4368fe1/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:15.239791 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea372f4d-1c07-4ff3-85ec-e629a11945cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d91979a6ce9468c990b9e08e267d329, user_id=6ed299381d5b49fb8577287af4368fe1, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:15.242709 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea372f4d-1c07-4ff3-85ec-e629a11945cd admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:15.262137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1158/2295] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:14:15 2022] PUT /identity/v3/projects/3d91979a6ce9468c990b9e08e267d329/users/6ed299381d5b49fb8577287af4368fe1/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:15.278102 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73561f4-9664-4ea1-9bc8-fb5465ff9fe3 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:15.278354 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73561f4-9664-4ea1-9bc8-fb5465ff9fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:15.278663 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73561f4-9664-4ea1-9bc8-fb5465ff9fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:15.311070 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-f73561f4-9664-4ea1-9bc8-fb5465ff9fe3 None None] MFA Rules not processed for user `6ed299381d5b49fb8577287af4368fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:15.312500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f73561f4-9664-4ea1-9bc8-fb5465ff9fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:15.323613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:15.388345 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1138/2296] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:15.403347 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea372f4d-1c07-4ff3-85ec-e629a11945cd admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:15.405207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ea372f4d-1c07-4ff3-85ec-e629a11945cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:15.469109 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea372f4d-1c07-4ff3-85ec-e629a11945cd admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:15.473926 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ca7ae-4f48-44bd-933f-80493f50396b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:15.474557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ca7ae-4f48-44bd-933f-80493f50396b service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:15.474817 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ca7ae-4f48-44bd-933f-80493f50396b service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:15.475086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ca7ae-4f48-44bd-933f-80493f50396b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:15.484741 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ca7ae-4f48-44bd-933f-80493f50396b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:15.486175 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ca7ae-4f48-44bd-933f-80493f50396b service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:15.495109 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1159/2297] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:14:16.886535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a3d9a-05ce-4f6c-8ee2-250c72c533e8 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:16.887108 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a3d9a-05ce-4f6c-8ee2-250c72c533e8 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:16.887108 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a3d9a-05ce-4f6c-8ee2-250c72c533e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:16.906090 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-636a3d9a-05ce-4f6c-8ee2-250c72c533e8 None None] MFA Rules not processed for user `6ed299381d5b49fb8577287af4368fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:16.907479 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-636a3d9a-05ce-4f6c-8ee2-250c72c533e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:16.915151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1139/2298] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:25.491029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=55,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:26.901203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=26 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:30.984240 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b10f84-23c5-4d3a-b5ab-08348f385fc7 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:30.984746 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b10f84-23c5-4d3a-b5ab-08348f385fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:30.984956 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b10f84-23c5-4d3a-b5ab-08348f385fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:31.017034 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-76b10f84-23c5-4d3a-b5ab-08348f385fc7 None None] MFA Rules not processed for user `471dc5c1db64411ab1d80ad6c038e519`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:31.018345 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-76b10f84-23c5-4d3a-b5ab-08348f385fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:31.078035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1160/2299] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:35.581377 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a3d9a-05ce-4f6c-8ee2-250c72c533e8 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:35.583300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-636a3d9a-05ce-4f6c-8ee2-250c72c533e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:35.662265 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a3d9a-05ce-4f6c-8ee2-250c72c533e8 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:35.668357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5a3ec-be4a-4272-90f1-3e0a1007e576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:35.669011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5a3ec-be4a-4272-90f1-3e0a1007e576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:35.669288 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5a3ec-be4a-4272-90f1-3e0a1007e576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:35.669547 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5a3ec-be4a-4272-90f1-3e0a1007e576 admin admin] PATH_INFO: `/v3/users/2573fe4a125049979ce42bfff8672a13` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:35.672214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5a3ec-be4a-4272-90f1-3e0a1007e576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2573fe4a125049979ce42bfff8672a13)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:35.674610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5a3ec-be4a-4272-90f1-3e0a1007e576 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:35.726745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-16a5a3ec-be4a-4272-90f1-3e0a1007e576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2573fe4a125049979ce42bfff8672a13'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:35.727404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-16a5a3ec-be4a-4272-90f1-3e0a1007e576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2573fe4a125049979ce42bfff8672a13'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:35.728030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-16a5a3ec-be4a-4272-90f1-3e0a1007e576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2573fe4a125049979ce42bfff8672a13'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:35.736589 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-16a5a3ec-be4a-4272-90f1-3e0a1007e576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2573fe4a125049979ce42bfff8672a13'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:35.755311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-16a5a3ec-be4a-4272-90f1-3e0a1007e576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2573fe4a125049979ce42bfff8672a13'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:35.770985 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1140/2300] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:14:35 2022] DELETE /identity/v3/users/2573fe4a125049979ce42bfff8672a13 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:35.775705 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced1634-e623-443b-815f-965a6c43b262 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:35.776209 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced1634-e623-443b-815f-965a6c43b262 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:35.776524 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced1634-e623-443b-815f-965a6c43b262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:35.796789 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-aced1634-e623-443b-815f-965a6c43b262 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:35.798192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-aced1634-e623-443b-815f-965a6c43b262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:35.864946 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1161/2301] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:35.874986 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5a3ec-be4a-4272-90f1-3e0a1007e576 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:35.876785 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-16a5a3ec-be4a-4272-90f1-3e0a1007e576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:35.969028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5a3ec-be4a-4272-90f1-3e0a1007e576 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:35.973674 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b585984-9e48-4b10-a9f0-d74bcb88495b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:35.974559 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b585984-9e48-4b10-a9f0-d74bcb88495b service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:35.974880 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b585984-9e48-4b10-a9f0-d74bcb88495b service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:35.975154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b585984-9e48-4b10-a9f0-d74bcb88495b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:35.986795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b585984-9e48-4b10-a9f0-d74bcb88495b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:35.988292 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b585984-9e48-4b10-a9f0-d74bcb88495b service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:36.001753 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1141/2302] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:14:36.565820 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced1634-e623-443b-815f-965a6c43b262 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:36.567552 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-aced1634-e623-443b-815f-965a6c43b262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:36.583393 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced1634-e623-443b-815f-965a6c43b262 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:36.589278 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e464b1-9a59-42f6-8388-08bd34a47662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:36.590879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e464b1-9a59-42f6-8388-08bd34a47662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:36.590879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e464b1-9a59-42f6-8388-08bd34a47662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:36.590879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e464b1-9a59-42f6-8388-08bd34a47662 admin admin] PATH_INFO: `/v3/projects/432b19722b6a4884a847872a3e181b49` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:36.592751 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e464b1-9a59-42f6-8388-08bd34a47662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=432b19722b6a4884a847872a3e181b49)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:36.594838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e464b1-9a59-42f6-8388-08bd34a47662 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:36.648429 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f9e464b1-9a59-42f6-8388-08bd34a47662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '432b19722b6a4884a847872a3e181b49', 'request_id': 'req-f9e464b1-9a59-42f6-8388-08bd34a47662'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:36.656449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f9e464b1-9a59-42f6-8388-08bd34a47662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '432b19722b6a4884a847872a3e181b49', 'request_id': 'req-f9e464b1-9a59-42f6-8388-08bd34a47662'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:36.666544 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f9e464b1-9a59-42f6-8388-08bd34a47662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '432b19722b6a4884a847872a3e181b49', 'request_id': 'req-f9e464b1-9a59-42f6-8388-08bd34a47662'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:36.688619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f9e464b1-9a59-42f6-8388-08bd34a47662 admin admin] The token cache is being invalidate because project 432b19722b6a4884a847872a3e181b49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:14:36.689025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f9e464b1-9a59-42f6-8388-08bd34a47662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:36.689947 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f9e464b1-9a59-42f6-8388-08bd34a47662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:36.691617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1162/2303] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:14:36 2022] DELETE /identity/v3/projects/432b19722b6a4884a847872a3e181b49 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:36.696707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38171ff-c395-495b-9ae8-da4098ba0599 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:36.697508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38171ff-c395-495b-9ae8-da4098ba0599 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:36.697982 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38171ff-c395-495b-9ae8-da4098ba0599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:36.725894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-b38171ff-c395-495b-9ae8-da4098ba0599 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:36.727130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b38171ff-c395-495b-9ae8-da4098ba0599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:36.745890 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:36.800221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1142/2304] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:14:36.802241 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653389f-a7d8-4eff-9353-63ba419dc2b7 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:36.802884 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653389f-a7d8-4eff-9353-63ba419dc2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:36.803163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653389f-a7d8-4eff-9353-63ba419dc2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:36.825465 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-4653389f-a7d8-4eff-9353-63ba419dc2b7 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:36.826742 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4653389f-a7d8-4eff-9353-63ba419dc2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:36.836049 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1163/2305] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:36.840639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38171ff-c395-495b-9ae8-da4098ba0599 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:36.851013 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38171ff-c395-495b-9ae8-da4098ba0599 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:36.857180 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb658209-f15f-4fc9-b801-c99e0306c943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:36.857818 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb658209-f15f-4fc9-b801-c99e0306c943 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:36.858104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb658209-f15f-4fc9-b801-c99e0306c943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:36.858379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb658209-f15f-4fc9-b801-c99e0306c943 admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:36.859045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb658209-f15f-4fc9-b801-c99e0306c943 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:36.859358 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb658209-f15f-4fc9-b801-c99e0306c943 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:36.861056 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb658209-f15f-4fc9-b801-c99e0306c943 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:36.875463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1143/2306] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:36.878508 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653389f-a7d8-4eff-9353-63ba419dc2b7 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:36.888819 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653389f-a7d8-4eff-9353-63ba419dc2b7 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:36.894078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8b9c0-14f8-44ba-bdd5-e158d9e97b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:36.894690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8b9c0-14f8-44ba-bdd5-e158d9e97b5d admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:36.895070 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8b9c0-14f8-44ba-bdd5-e158d9e97b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:36.895222 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8b9c0-14f8-44ba-bdd5-e158d9e97b5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:36.895921 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8b9c0-14f8-44ba-bdd5-e158d9e97b5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:36.898236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8b9c0-14f8-44ba-bdd5-e158d9e97b5d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:36.928604 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:36.937614 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1164/2307] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:14:36 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:36.941970 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb658209-f15f-4fc9-b801-c99e0306c943 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:36.956692 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb658209-f15f-4fc9-b801-c99e0306c943 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:37.074095 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebcf0a2-f637-4edd-b82d-40baaa08af97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:37.074891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcf0a2-f637-4edd-b82d-40baaa08af97 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.075240 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcf0a2-f637-4edd-b82d-40baaa08af97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.076240 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcf0a2-f637-4edd-b82d-40baaa08af97 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.076359 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebcf0a2-f637-4edd-b82d-40baaa08af97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:37.078451 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebcf0a2-f637-4edd-b82d-40baaa08af97 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:37.104953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1144/2308] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:14:36 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:37.111566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8b9c0-14f8-44ba-bdd5-e158d9e97b5d admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:37.123004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8b9c0-14f8-44ba-bdd5-e158d9e97b5d admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:37.128223 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929fade-86a1-4f26-a1d0-d63a311d18c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:37.128833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929fade-86a1-4f26-a1d0-d63a311d18c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.129107 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929fade-86a1-4f26-a1d0-d63a311d18c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.129374 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929fade-86a1-4f26-a1d0-d63a311d18c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.131087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3929fade-86a1-4f26-a1d0-d63a311d18c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:37.132502 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3929fade-86a1-4f26-a1d0-d63a311d18c0 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:37.154339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1165/2309] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:37.155567 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebcf0a2-f637-4edd-b82d-40baaa08af97 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:37.166129 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebcf0a2-f637-4edd-b82d-40baaa08af97 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:37.171643 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbd683-9451-4153-964f-9c8e14c75f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:37.172791 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbd683-9451-4153-964f-9c8e14c75f07 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.173071 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbd683-9451-4153-964f-9c8e14c75f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.173348 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbd683-9451-4153-964f-9c8e14c75f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.174058 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbd683-9451-4153-964f-9c8e14c75f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:37.175999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbd683-9451-4153-964f-9c8e14c75f07 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:37.188605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1145/2310] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:37.192927 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929fade-86a1-4f26-a1d0-d63a311d18c0 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:37.205478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929fade-86a1-4f26-a1d0-d63a311d18c0 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:37.210630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e084f7-c32f-487c-ae5e-acc18f2e509f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:37.211283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e084f7-c32f-487c-ae5e-acc18f2e509f admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.211652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e084f7-c32f-487c-ae5e-acc18f2e509f admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.211925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e084f7-c32f-487c-ae5e-acc18f2e509f admin admin] PATH_INFO: `/v3/projects/ce36259bd4394aad8b5ae74a1234fbd6/users/495af225863546398f002e4361fa6fb4/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.234731 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e084f7-c32f-487c-ae5e-acc18f2e509f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce36259bd4394aad8b5ae74a1234fbd6, user_id=495af225863546398f002e4361fa6fb4, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:37.237732 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e084f7-c32f-487c-ae5e-acc18f2e509f admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:37.254819 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1166/2311] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:14:37 2022] PUT /identity/v3/projects/ce36259bd4394aad8b5ae74a1234fbd6/users/495af225863546398f002e4361fa6fb4/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:37.267083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042e839-fa22-428c-bf89-042630aad1c6 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.267387 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042e839-fa22-428c-bf89-042630aad1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.267690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042e839-fa22-428c-bf89-042630aad1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.303961 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-5042e839-fa22-428c-bf89-042630aad1c6 None None] MFA Rules not processed for user `495af225863546398f002e4361fa6fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:37.305212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-5042e839-fa22-428c-bf89-042630aad1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:37.384820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1146/2312] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:37.388998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e084f7-c32f-487c-ae5e-acc18f2e509f admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:37.461657 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e084f7-c32f-487c-ae5e-acc18f2e509f admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:37.467231 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1be8e2-88f6-46cf-9af1-66bb45d5ff9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:37.467929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1be8e2-88f6-46cf-9af1-66bb45d5ff9e admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.468212 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1be8e2-88f6-46cf-9af1-66bb45d5ff9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.468490 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1be8e2-88f6-46cf-9af1-66bb45d5ff9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.469201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1be8e2-88f6-46cf-9af1-66bb45d5ff9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:37.471224 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1be8e2-88f6-46cf-9af1-66bb45d5ff9e admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:37.501988 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1167/2313] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:14:37 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:14:37.505955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042e839-fa22-428c-bf89-042630aad1c6 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:37.518602 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042e839-fa22-428c-bf89-042630aad1c6 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:37.565397 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:37.656873 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd934675-ec62-4704-bd9d-e5eeeb66e345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:37.656873 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd934675-ec62-4704-bd9d-e5eeeb66e345 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.656873 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd934675-ec62-4704-bd9d-e5eeeb66e345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.656873 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd934675-ec62-4704-bd9d-e5eeeb66e345 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.657359 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd934675-ec62-4704-bd9d-e5eeeb66e345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:37.658313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd934675-ec62-4704-bd9d-e5eeeb66e345 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:37.696607 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1147/2314] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:14:37 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:14:37.700685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1be8e2-88f6-46cf-9af1-66bb45d5ff9e admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:37.712112 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1be8e2-88f6-46cf-9af1-66bb45d5ff9e admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:37.717677 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dbc36-6927-40e0-8158-9405193fb394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:37.719761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dbc36-6927-40e0-8158-9405193fb394 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.720025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dbc36-6927-40e0-8158-9405193fb394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.720407 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dbc36-6927-40e0-8158-9405193fb394 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.721605 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dbc36-6927-40e0-8158-9405193fb394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:37.723283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dbc36-6927-40e0-8158-9405193fb394 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:37.738983 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1168/2315] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:14:37.743292 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd934675-ec62-4704-bd9d-e5eeeb66e345 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:37.758099 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd934675-ec62-4704-bd9d-e5eeeb66e345 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:37.765078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773fa0d7-d1bd-4258-adea-05f2e1107a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:37.765889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773fa0d7-d1bd-4258-adea-05f2e1107a0c admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.766224 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773fa0d7-d1bd-4258-adea-05f2e1107a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.766552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773fa0d7-d1bd-4258-adea-05f2e1107a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.767253 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773fa0d7-d1bd-4258-adea-05f2e1107a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:37.769022 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773fa0d7-d1bd-4258-adea-05f2e1107a0c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:37.781107 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1148/2316] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:14:37.785420 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dbc36-6927-40e0-8158-9405193fb394 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:37.796218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dbc36-6927-40e0-8158-9405193fb394 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:37.801357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e7776-0097-453e-918d-869aa4f21170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:37.802117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e7776-0097-453e-918d-869aa4f21170 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.802402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e7776-0097-453e-918d-869aa4f21170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.802986 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e7776-0097-453e-918d-869aa4f21170 admin admin] PATH_INFO: `/v3/projects/a15a28638b4b49e382ab0cb9f3175fd5/users/f430f6e0afbb46faaf78ff616383f7e2/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.822603 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e7776-0097-453e-918d-869aa4f21170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a15a28638b4b49e382ab0cb9f3175fd5, user_id=f430f6e0afbb46faaf78ff616383f7e2, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:37.825520 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e7776-0097-453e-918d-869aa4f21170 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:37.842522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1169/2317] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:14:37 2022] PUT /identity/v3/projects/a15a28638b4b49e382ab0cb9f3175fd5/users/f430f6e0afbb46faaf78ff616383f7e2/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:37.853132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f2bf3-aec6-43c0-ac6a-7b40e69ca180 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:37.856680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f2bf3-aec6-43c0-ac6a-7b40e69ca180 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:37.857230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f2bf3-aec6-43c0-ac6a-7b40e69ca180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:37.890125 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-6c1f2bf3-aec6-43c0-ac6a-7b40e69ca180 None None] MFA Rules not processed for user `f430f6e0afbb46faaf78ff616383f7e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:37.891431 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-6c1f2bf3-aec6-43c0-ac6a-7b40e69ca180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:37.971933 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1149/2318] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:37.972809 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e7776-0097-453e-918d-869aa4f21170 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:38.039790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e7776-0097-453e-918d-869aa4f21170 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:38.044912 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac8b90-2c21-4341-b41b-1a7974eaee68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:38.045696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac8b90-2c21-4341-b41b-1a7974eaee68 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:38.045979 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac8b90-2c21-4341-b41b-1a7974eaee68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:38.046248 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac8b90-2c21-4341-b41b-1a7974eaee68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:38.046887 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac8b90-2c21-4341-b41b-1a7974eaee68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:38.049020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac8b90-2c21-4341-b41b-1a7974eaee68 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:38.100578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:38.109502 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1170/2319] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:14:37 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:14:38.113904 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f2bf3-aec6-43c0-ac6a-7b40e69ca180 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:38.131941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f2bf3-aec6-43c0-ac6a-7b40e69ca180 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:38.256910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c7209-8c6e-4113-b9a2-1f19a1c2bb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:38.260033 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c7209-8c6e-4113-b9a2-1f19a1c2bb31 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:38.260303 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c7209-8c6e-4113-b9a2-1f19a1c2bb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:38.260576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c7209-8c6e-4113-b9a2-1f19a1c2bb31 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:38.261305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c7209-8c6e-4113-b9a2-1f19a1c2bb31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:38.264150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c7209-8c6e-4113-b9a2-1f19a1c2bb31 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:38.305543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1150/2320] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:14:38 2022] POST /identity/v3/users => generated 299 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:38.309895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac8b90-2c21-4341-b41b-1a7974eaee68 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:38.323841 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac8b90-2c21-4341-b41b-1a7974eaee68 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:38.334726 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1872a9fc-a992-444b-a24d-25769eb7c0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:38.335482 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1872a9fc-a992-444b-a24d-25769eb7c0c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:38.335760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1872a9fc-a992-444b-a24d-25769eb7c0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:38.336074 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1872a9fc-a992-444b-a24d-25769eb7c0c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:38.336736 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1872a9fc-a992-444b-a24d-25769eb7c0c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:38.338567 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1872a9fc-a992-444b-a24d-25769eb7c0c5 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:38.360322 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1171/2321] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:38.364284 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c7209-8c6e-4113-b9a2-1f19a1c2bb31 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:38.377403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c7209-8c6e-4113-b9a2-1f19a1c2bb31 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:38.382761 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415488d-d806-4386-a115-8cdc5cecc673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:38.383618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415488d-d806-4386-a115-8cdc5cecc673 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:38.383957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415488d-d806-4386-a115-8cdc5cecc673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:38.384246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415488d-d806-4386-a115-8cdc5cecc673 admin admin] PATH_INFO: `/v3/projects/6f51fdc6654c4332b38f7589dea95f70/users/a18ba39d160e4ba3ad83d0a8002028c4/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:38.410081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415488d-d806-4386-a115-8cdc5cecc673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f51fdc6654c4332b38f7589dea95f70, user_id=a18ba39d160e4ba3ad83d0a8002028c4, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:38.411231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:38.416474 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415488d-d806-4386-a115-8cdc5cecc673 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:38.446638 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1151/2322] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:14:38 2022] PUT /identity/v3/projects/6f51fdc6654c4332b38f7589dea95f70/users/a18ba39d160e4ba3ad83d0a8002028c4/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:38.449226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1872a9fc-a992-444b-a24d-25769eb7c0c5 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:38.467665 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1872a9fc-a992-444b-a24d-25769eb7c0c5 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:38.538055 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769d052-4337-4e0d-ad0d-bdabda39e75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:38.538675 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769d052-4337-4e0d-ad0d-bdabda39e75d admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:38.538952 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769d052-4337-4e0d-ad0d-bdabda39e75d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:38.539221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769d052-4337-4e0d-ad0d-bdabda39e75d admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:38.539904 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769d052-4337-4e0d-ad0d-bdabda39e75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:38.541645 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769d052-4337-4e0d-ad0d-bdabda39e75d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:38.551975 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1172/2323] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:38.556491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415488d-d806-4386-a115-8cdc5cecc673 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:38.567924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415488d-d806-4386-a115-8cdc5cecc673 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:38.573430 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72d958-d64f-4647-8acd-1a62cb0956ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:38.574206 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72d958-d64f-4647-8acd-1a62cb0956ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:38.574594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72d958-d64f-4647-8acd-1a62cb0956ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:38.574961 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72d958-d64f-4647-8acd-1a62cb0956ed admin admin] PATH_INFO: `/v3/projects/6f51fdc6654c4332b38f7589dea95f70/users/a18ba39d160e4ba3ad83d0a8002028c4/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:38.577035 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72d958-d64f-4647-8acd-1a62cb0956ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f51fdc6654c4332b38f7589dea95f70, user_id=a18ba39d160e4ba3ad83d0a8002028c4, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:38.580812 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72d958-d64f-4647-8acd-1a62cb0956ed admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:38.594662 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1152/2324] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:14:38 2022] PUT /identity/v3/projects/6f51fdc6654c4332b38f7589dea95f70/users/a18ba39d160e4ba3ad83d0a8002028c4/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:38.605091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a790bf-2289-468a-93b5-9b9d6aab5605 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:38.605385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a790bf-2289-468a-93b5-9b9d6aab5605 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:38.605743 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a790bf-2289-468a-93b5-9b9d6aab5605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:38.638189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-c3a790bf-2289-468a-93b5-9b9d6aab5605 None None] MFA Rules not processed for user `a18ba39d160e4ba3ad83d0a8002028c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:38.639780 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c3a790bf-2289-468a-93b5-9b9d6aab5605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:38.729798 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1173/2325] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:14:38.742072 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72d958-d64f-4647-8acd-1a62cb0956ed admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:38.744520 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8a72d958-d64f-4647-8acd-1a62cb0956ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:38.818875 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72d958-d64f-4647-8acd-1a62cb0956ed admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:38.822779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33f43f-ca4d-4584-b1be-7ac17beed226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:38.823401 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33f43f-ca4d-4584-b1be-7ac17beed226 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:38.823736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33f43f-ca4d-4584-b1be-7ac17beed226 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:38.824023 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33f43f-ca4d-4584-b1be-7ac17beed226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:38.882008 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33f43f-ca4d-4584-b1be-7ac17beed226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:38.883503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33f43f-ca4d-4584-b1be-7ac17beed226 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:38.892607 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1153/2326] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:14:42.527824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a790bf-2289-468a-93b5-9b9d6aab5605 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:42.529663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c3a790bf-2289-468a-93b5-9b9d6aab5605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:42.542401 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a790bf-2289-468a-93b5-9b9d6aab5605 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:42.547541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa2ef4-a160-4d8a-bbc7-93f5aeab3f0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:42.548150 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa2ef4-a160-4d8a-bbc7-93f5aeab3f0a service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:42.548430 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa2ef4-a160-4d8a-bbc7-93f5aeab3f0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:42.548699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa2ef4-a160-4d8a-bbc7-93f5aeab3f0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:42.610013 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa2ef4-a160-4d8a-bbc7-93f5aeab3f0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:42.611559 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa2ef4-a160-4d8a-bbc7-93f5aeab3f0a service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:42.621046 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1174/2327] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:14:43.389038 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33f43f-ca4d-4584-b1be-7ac17beed226 service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:43.399203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33f43f-ca4d-4584-b1be-7ac17beed226 service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:43.406951 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe40c3e-0220-4f0b-b7a6-02ecbd0e31ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:43.408784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe40c3e-0220-4f0b-b7a6-02ecbd0e31ca service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:43.409067 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe40c3e-0220-4f0b-b7a6-02ecbd0e31ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:43.409343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe40c3e-0220-4f0b-b7a6-02ecbd0e31ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:43.422658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe40c3e-0220-4f0b-b7a6-02ecbd0e31ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:43.425439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe40c3e-0220-4f0b-b7a6-02ecbd0e31ca service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:43.442121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1154/2328] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:14:46.528225 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df0283-ccd4-48be-9a4f-9e61011478b7 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:46.528771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df0283-ccd4-48be-9a4f-9e61011478b7 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:46.528771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df0283-ccd4-48be-9a4f-9e61011478b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:46.548281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-c8df0283-ccd4-48be-9a4f-9e61011478b7 None None] MFA Rules not processed for user `64d54cb95b264c18b45d90822507fae1`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:46.549585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c8df0283-ccd4-48be-9a4f-9e61011478b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:46.553790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:46.607138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1175/2329] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:48.806248 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe40c3e-0220-4f0b-b7a6-02ecbd0e31ca service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:48.807886 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9fe40c3e-0220-4f0b-b7a6-02ecbd0e31ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:48.879800 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe40c3e-0220-4f0b-b7a6-02ecbd0e31ca service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:48.884999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b2399-6412-48f4-a4fe-435c6ae7d05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:48.886198 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b2399-6412-48f4-a4fe-435c6ae7d05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:48.886476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b2399-6412-48f4-a4fe-435c6ae7d05e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:48.886761 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b2399-6412-48f4-a4fe-435c6ae7d05e admin admin] PATH_INFO: `/v3/users/df1ac13407df46088eaf031e7c12ec3a` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:48.889884 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b2399-6412-48f4-a4fe-435c6ae7d05e admin admin] RBAC: Authorizing `identity:delete_user(user_id=df1ac13407df46088eaf031e7c12ec3a)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:48.891913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b2399-6412-48f4-a4fe-435c6ae7d05e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:48.923936 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:48.960890 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-433b2399-6412-48f4-a4fe-435c6ae7d05e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df1ac13407df46088eaf031e7c12ec3a'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:48.961774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-433b2399-6412-48f4-a4fe-435c6ae7d05e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df1ac13407df46088eaf031e7c12ec3a'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:48.962648 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-433b2399-6412-48f4-a4fe-435c6ae7d05e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df1ac13407df46088eaf031e7c12ec3a'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:48.971882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-433b2399-6412-48f4-a4fe-435c6ae7d05e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df1ac13407df46088eaf031e7c12ec3a'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:48.997984 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-433b2399-6412-48f4-a4fe-435c6ae7d05e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df1ac13407df46088eaf031e7c12ec3a'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:49.019690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1155/2330] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:14:48 2022] DELETE /identity/v3/users/df1ac13407df46088eaf031e7c12ec3a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:49.023066 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4371c9-e505-4c76-815c-5a44bab01a28 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:49.023585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4371c9-e505-4c76-815c-5a44bab01a28 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:49.023985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4371c9-e505-4c76-815c-5a44bab01a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:49.045339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-cd4371c9-e505-4c76-815c-5a44bab01a28 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:49.046710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-cd4371c9-e505-4c76-815c-5a44bab01a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:49.130979 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1176/2331] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:49.143279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b2399-6412-48f4-a4fe-435c6ae7d05e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:49.150971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-433b2399-6412-48f4-a4fe-435c6ae7d05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:49.239539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b2399-6412-48f4-a4fe-435c6ae7d05e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:49.244692 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f2c6e-4604-4c9f-80d2-6cdce3fb9c2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:49.245426 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f2c6e-4604-4c9f-80d2-6cdce3fb9c2c service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:49.245904 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f2c6e-4604-4c9f-80d2-6cdce3fb9c2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:49.246361 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f2c6e-4604-4c9f-80d2-6cdce3fb9c2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:49.264717 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f2c6e-4604-4c9f-80d2-6cdce3fb9c2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:49.266237 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f2c6e-4604-4c9f-80d2-6cdce3fb9c2c service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:49.278656 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1156/2332] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:14:49.581024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4371c9-e505-4c76-815c-5a44bab01a28 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:49.584761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-cd4371c9-e505-4c76-815c-5a44bab01a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:49.621697 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4371c9-e505-4c76-815c-5a44bab01a28 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:49.628110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2feec-2e8f-4679-9b25-c6e79b962ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:49.628857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2feec-2e8f-4679-9b25-c6e79b962ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:49.629350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2feec-2e8f-4679-9b25-c6e79b962ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:49.630256 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2feec-2e8f-4679-9b25-c6e79b962ce6 admin admin] PATH_INFO: `/v3/projects/cc42282f6bce4842bc40e07475315514` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:49.631038 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2feec-2e8f-4679-9b25-c6e79b962ce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc42282f6bce4842bc40e07475315514)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:49.633153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2feec-2e8f-4679-9b25-c6e79b962ce6 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:49.692004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e7a2feec-2e8f-4679-9b25-c6e79b962ce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc42282f6bce4842bc40e07475315514', 'request_id': 'req-e7a2feec-2e8f-4679-9b25-c6e79b962ce6'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:49.699105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e7a2feec-2e8f-4679-9b25-c6e79b962ce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc42282f6bce4842bc40e07475315514', 'request_id': 'req-e7a2feec-2e8f-4679-9b25-c6e79b962ce6'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:49.709937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e7a2feec-2e8f-4679-9b25-c6e79b962ce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc42282f6bce4842bc40e07475315514', 'request_id': 'req-e7a2feec-2e8f-4679-9b25-c6e79b962ce6'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:49.722774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e7a2feec-2e8f-4679-9b25-c6e79b962ce6 admin admin] The token cache is being invalidate because project cc42282f6bce4842bc40e07475315514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:14:49.723062 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e7a2feec-2e8f-4679-9b25-c6e79b962ce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:49.723722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e7a2feec-2e8f-4679-9b25-c6e79b962ce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:49.725228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1177/2333] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:14:49 2022] DELETE /identity/v3/projects/cc42282f6bce4842bc40e07475315514 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:49.731138 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e5d6b-cf24-45b5-ab1c-1651dcc4e8b0 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:49.731438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e5d6b-cf24-45b5-ab1c-1651dcc4e8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:49.731746 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e5d6b-cf24-45b5-ab1c-1651dcc4e8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:49.756729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-185e5d6b-cf24-45b5-ab1c-1651dcc4e8b0 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:49.758111 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-185e5d6b-cf24-45b5-ab1c-1651dcc4e8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:49.822691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1157/2334] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:49.830654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af59488-20ae-408c-ab73-b64a298757c8 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:49.830940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af59488-20ae-408c-ab73-b64a298757c8 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:49.831215 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af59488-20ae-408c-ab73-b64a298757c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:49.852202 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-0af59488-20ae-408c-ab73-b64a298757c8 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:49.853419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-0af59488-20ae-408c-ab73-b64a298757c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:49.861723 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1178/2335] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:49.866119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e5d6b-cf24-45b5-ab1c-1651dcc4e8b0 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:49.876572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e5d6b-cf24-45b5-ab1c-1651dcc4e8b0 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:49.881488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824c920-b786-490c-8be6-9118a9fa2592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:49.882105 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824c920-b786-490c-8be6-9118a9fa2592 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:49.882386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824c920-b786-490c-8be6-9118a9fa2592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:49.882659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824c920-b786-490c-8be6-9118a9fa2592 admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:49.883359 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824c920-b786-490c-8be6-9118a9fa2592 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:49.883696 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824c920-b786-490c-8be6-9118a9fa2592 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:49.885339 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824c920-b786-490c-8be6-9118a9fa2592 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:49.898029 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1158/2336] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:49.902477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af59488-20ae-408c-ab73-b64a298757c8 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:49.912847 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af59488-20ae-408c-ab73-b64a298757c8 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:49.917625 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff391d-02ff-4970-b838-7aa2a1336e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:49.918931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff391d-02ff-4970-b838-7aa2a1336e40 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:49.919400 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff391d-02ff-4970-b838-7aa2a1336e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:49.919703 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff391d-02ff-4970-b838-7aa2a1336e40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:49.920358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff391d-02ff-4970-b838-7aa2a1336e40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:49.922291 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff391d-02ff-4970-b838-7aa2a1336e40 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:49.953685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1179/2337] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:14:49 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:14:49.958328 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824c920-b786-490c-8be6-9118a9fa2592 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:49.969436 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824c920-b786-490c-8be6-9118a9fa2592 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:50.000954 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:50.049270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d34875-5a5d-4b82-a097-09ec3baec7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:50.049890 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d34875-5a5d-4b82-a097-09ec3baec7d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:50.050164 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d34875-5a5d-4b82-a097-09ec3baec7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:50.050435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d34875-5a5d-4b82-a097-09ec3baec7d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:50.051060 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d34875-5a5d-4b82-a097-09ec3baec7d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:50.053190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d34875-5a5d-4b82-a097-09ec3baec7d7 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:50.077572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1159/2338] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:14:49 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:50.081692 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff391d-02ff-4970-b838-7aa2a1336e40 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:50.104615 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff391d-02ff-4970-b838-7aa2a1336e40 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:50.111141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261ef49-9a4d-4e84-82db-85bb8dca20f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:50.112072 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261ef49-9a4d-4e84-82db-85bb8dca20f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:50.112351 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261ef49-9a4d-4e84-82db-85bb8dca20f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:50.112619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261ef49-9a4d-4e84-82db-85bb8dca20f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:50.113263 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a261ef49-9a4d-4e84-82db-85bb8dca20f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:50.114955 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a261ef49-9a4d-4e84-82db-85bb8dca20f2 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:50.117625 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:50.130659 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1180/2339] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:50.132161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d34875-5a5d-4b82-a097-09ec3baec7d7 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:50.143160 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d34875-5a5d-4b82-a097-09ec3baec7d7 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:50.150702 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e7b2f-96e6-4962-a398-6910f02a76d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:50.151389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e7b2f-96e6-4962-a398-6910f02a76d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:50.151739 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e7b2f-96e6-4962-a398-6910f02a76d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:50.152081 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e7b2f-96e6-4962-a398-6910f02a76d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:50.152922 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e7b2f-96e6-4962-a398-6910f02a76d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:50.154911 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e7b2f-96e6-4962-a398-6910f02a76d5 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:50.165887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1160/2340] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:50.170456 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261ef49-9a4d-4e84-82db-85bb8dca20f2 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:50.180478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261ef49-9a4d-4e84-82db-85bb8dca20f2 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:50.185651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb8fbe-620a-4dc4-80d1-d1ece9cb44e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:50.186348 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb8fbe-620a-4dc4-80d1-d1ece9cb44e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:50.186564 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb8fbe-620a-4dc4-80d1-d1ece9cb44e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:50.187056 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb8fbe-620a-4dc4-80d1-d1ece9cb44e7 admin admin] PATH_INFO: `/v3/projects/f38841ae9e5b4cbf8344994a1335eb42/users/dd3a243d392c452cb5271554d6d964a3/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:50.204784 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb8fbe-620a-4dc4-80d1-d1ece9cb44e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f38841ae9e5b4cbf8344994a1335eb42, user_id=dd3a243d392c452cb5271554d6d964a3, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:50.208115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb8fbe-620a-4dc4-80d1-d1ece9cb44e7 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:50.224894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1181/2341] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:14:50 2022] PUT /identity/v3/projects/f38841ae9e5b4cbf8344994a1335eb42/users/dd3a243d392c452cb5271554d6d964a3/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:50.236958 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223116b-e1bd-4015-bd03-59305339fe1e None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:50.237249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223116b-e1bd-4015-bd03-59305339fe1e None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:50.237525 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223116b-e1bd-4015-bd03-59305339fe1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:50.273471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-f223116b-e1bd-4015-bd03-59305339fe1e None None] MFA Rules not processed for user `dd3a243d392c452cb5271554d6d964a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:50.275195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f223116b-e1bd-4015-bd03-59305339fe1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:50.374384 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1161/2342] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:50.385452 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb8fbe-620a-4dc4-80d1-d1ece9cb44e7 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:50.387160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-f5cb8fbe-620a-4dc4-80d1-d1ece9cb44e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:50.447944 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb8fbe-620a-4dc4-80d1-d1ece9cb44e7 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:50.451796 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a1a30-37a2-4040-baa1-2b39b4aaa966 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:50.452450 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a1a30-37a2-4040-baa1-2b39b4aaa966 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:50.452725 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a1a30-37a2-4040-baa1-2b39b4aaa966 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:50.452989 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a1a30-37a2-4040-baa1-2b39b4aaa966 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:50.462149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a1a30-37a2-4040-baa1-2b39b4aaa966 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:50.463562 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a1a30-37a2-4040-baa1-2b39b4aaa966 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:50.474489 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1182/2343] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:14:50.500195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223116b-e1bd-4015-bd03-59305339fe1e None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:50.566398 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223116b-e1bd-4015-bd03-59305339fe1e None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:50.571270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93347f9-31ee-4e0f-be42-7638ea4b8500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:50.572096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93347f9-31ee-4e0f-be42-7638ea4b8500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:50.572445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93347f9-31ee-4e0f-be42-7638ea4b8500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:50.572644 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93347f9-31ee-4e0f-be42-7638ea4b8500 admin admin] PATH_INFO: `/v3/users/dd3a243d392c452cb5271554d6d964a3` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:50.573789 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93347f9-31ee-4e0f-be42-7638ea4b8500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd3a243d392c452cb5271554d6d964a3)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:50.575882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93347f9-31ee-4e0f-be42-7638ea4b8500 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:50.631152 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:50.645516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d93347f9-31ee-4e0f-be42-7638ea4b8500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd3a243d392c452cb5271554d6d964a3'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:50.646370 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d93347f9-31ee-4e0f-be42-7638ea4b8500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd3a243d392c452cb5271554d6d964a3'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:50.647020 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d93347f9-31ee-4e0f-be42-7638ea4b8500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd3a243d392c452cb5271554d6d964a3'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:50.655292 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d93347f9-31ee-4e0f-be42-7638ea4b8500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd3a243d392c452cb5271554d6d964a3'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:50.667620 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d93347f9-31ee-4e0f-be42-7638ea4b8500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd3a243d392c452cb5271554d6d964a3'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:50.682776 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1162/2344] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:14:50 2022] DELETE /identity/v3/users/dd3a243d392c452cb5271554d6d964a3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:50.688230 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601036e-b96d-476d-ae18-07310ae0ea0a None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:50.688645 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601036e-b96d-476d-ae18-07310ae0ea0a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:50.689011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601036e-b96d-476d-ae18-07310ae0ea0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:50.697372 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93347f9-31ee-4e0f-be42-7638ea4b8500 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:50.699106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d93347f9-31ee-4e0f-be42-7638ea4b8500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:50.722926 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-e601036e-b96d-476d-ae18-07310ae0ea0a None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:50.728749 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e601036e-b96d-476d-ae18-07310ae0ea0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:50.787352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93347f9-31ee-4e0f-be42-7638ea4b8500 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:50.793472 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84aac5-23db-48d8-8e40-dabc19a3bb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:50.794487 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84aac5-23db-48d8-8e40-dabc19a3bb34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:50.799811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84aac5-23db-48d8-8e40-dabc19a3bb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:50.800333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84aac5-23db-48d8-8e40-dabc19a3bb34 admin admin] PATH_INFO: `/v3/users/495af225863546398f002e4361fa6fb4` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:50.801788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84aac5-23db-48d8-8e40-dabc19a3bb34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=495af225863546398f002e4361fa6fb4)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:50.803807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84aac5-23db-48d8-8e40-dabc19a3bb34 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:50.816825 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1183/2345] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:14:50.828272 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601036e-b96d-476d-ae18-07310ae0ea0a None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:50.829757 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e601036e-b96d-476d-ae18-07310ae0ea0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:50.862623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1f84aac5-23db-48d8-8e40-dabc19a3bb34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '495af225863546398f002e4361fa6fb4'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:50.863280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1f84aac5-23db-48d8-8e40-dabc19a3bb34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '495af225863546398f002e4361fa6fb4'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:50.863930 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1f84aac5-23db-48d8-8e40-dabc19a3bb34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '495af225863546398f002e4361fa6fb4'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:50.870955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1f84aac5-23db-48d8-8e40-dabc19a3bb34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '495af225863546398f002e4361fa6fb4'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:50.883861 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1f84aac5-23db-48d8-8e40-dabc19a3bb34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '495af225863546398f002e4361fa6fb4'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:50.888614 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601036e-b96d-476d-ae18-07310ae0ea0a None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:50.893186 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4856f9-783c-4ae3-bd1f-a7da4b858a57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:50.893793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4856f9-783c-4ae3-bd1f-a7da4b858a57 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:50.894067 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4856f9-783c-4ae3-bd1f-a7da4b858a57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:50.894345 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4856f9-783c-4ae3-bd1f-a7da4b858a57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:50.896104 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8f4856f9-783c-4ae3-bd1f-a7da4b858a57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:50.898728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1163/2346] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:14:50 2022] DELETE /identity/v3/users/495af225863546398f002e4361fa6fb4 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:50.903309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773f1d2-8f87-4bd1-b57b-b715c354233b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:50.903654 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773f1d2-8f87-4bd1-b57b-b715c354233b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:50.903931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773f1d2-8f87-4bd1-b57b-b715c354233b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:50.908397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4856f9-783c-4ae3-bd1f-a7da4b858a57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:50.909962 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4856f9-783c-4ae3-bd1f-a7da4b858a57 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:50.921138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1184/2347] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:14:50.922952 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-2773f1d2-8f87-4bd1-b57b-b715c354233b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:50.924196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2773f1d2-8f87-4bd1-b57b-b715c354233b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:50.954545 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:50.988336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1164/2348] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:50.998506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4856f9-783c-4ae3-bd1f-a7da4b858a57 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:50.999980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8f4856f9-783c-4ae3-bd1f-a7da4b858a57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:51.010811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4856f9-783c-4ae3-bd1f-a7da4b858a57 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:51.028597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:51.064052 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593b4dd-f5d0-46cb-99ea-aad28a41ff83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:51.064708 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593b4dd-f5d0-46cb-99ea-aad28a41ff83 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:51.064941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593b4dd-f5d0-46cb-99ea-aad28a41ff83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:51.065214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593b4dd-f5d0-46cb-99ea-aad28a41ff83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:51.077765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593b4dd-f5d0-46cb-99ea-aad28a41ff83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:51.079283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593b4dd-f5d0-46cb-99ea-aad28a41ff83 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:51.090126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1185/2349] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:14:51.402112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773f1d2-8f87-4bd1-b57b-b715c354233b None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:51.406350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2773f1d2-8f87-4bd1-b57b-b715c354233b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:51.426342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773f1d2-8f87-4bd1-b57b-b715c354233b None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:51.437652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db433261-fad8-4fb6-b0c3-40f8a2a06a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:51.438327 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db433261-fad8-4fb6-b0c3-40f8a2a06a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:51.438879 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db433261-fad8-4fb6-b0c3-40f8a2a06a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:51.439163 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db433261-fad8-4fb6-b0c3-40f8a2a06a42 admin admin] PATH_INFO: `/v3/projects/f38841ae9e5b4cbf8344994a1335eb42` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:51.442558 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db433261-fad8-4fb6-b0c3-40f8a2a06a42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f38841ae9e5b4cbf8344994a1335eb42)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:51.444597 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db433261-fad8-4fb6-b0c3-40f8a2a06a42 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:51.538076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-db433261-fad8-4fb6-b0c3-40f8a2a06a42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f38841ae9e5b4cbf8344994a1335eb42', 'request_id': 'req-db433261-fad8-4fb6-b0c3-40f8a2a06a42'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:51.546193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-db433261-fad8-4fb6-b0c3-40f8a2a06a42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f38841ae9e5b4cbf8344994a1335eb42', 'request_id': 'req-db433261-fad8-4fb6-b0c3-40f8a2a06a42'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:51.557301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-db433261-fad8-4fb6-b0c3-40f8a2a06a42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f38841ae9e5b4cbf8344994a1335eb42', 'request_id': 'req-db433261-fad8-4fb6-b0c3-40f8a2a06a42'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:51.573414 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-db433261-fad8-4fb6-b0c3-40f8a2a06a42 admin admin] The token cache is being invalidate because project f38841ae9e5b4cbf8344994a1335eb42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:14:51.573987 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-db433261-fad8-4fb6-b0c3-40f8a2a06a42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:51.587599 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-db433261-fad8-4fb6-b0c3-40f8a2a06a42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:51.590198 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1165/2350] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:14:51 2022] DELETE /identity/v3/projects/f38841ae9e5b4cbf8344994a1335eb42 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:51.594833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f696f3c-3cd3-4625-a534-1dfe235c3d3d None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:51.595114 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f696f3c-3cd3-4625-a534-1dfe235c3d3d None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:51.595387 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f696f3c-3cd3-4625-a534-1dfe235c3d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:51.625207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-4f696f3c-3cd3-4625-a534-1dfe235c3d3d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:51.626428 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4f696f3c-3cd3-4625-a534-1dfe235c3d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:51.718726 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1186/2351] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:51.726872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d289f0-65e1-4927-b5dd-6ce9e9492888 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:51.727161 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d289f0-65e1-4927-b5dd-6ce9e9492888 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:51.727465 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d289f0-65e1-4927-b5dd-6ce9e9492888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:51.755996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-b4d289f0-65e1-4927-b5dd-6ce9e9492888 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:51.757235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b4d289f0-65e1-4927-b5dd-6ce9e9492888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:51.769194 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1166/2352] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:51.773241 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f696f3c-3cd3-4625-a534-1dfe235c3d3d None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:51.786060 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f696f3c-3cd3-4625-a534-1dfe235c3d3d None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:51.793409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734ab58-d847-4a57-af62-16f38474609c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:51.794487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734ab58-d847-4a57-af62-16f38474609c admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:51.794806 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734ab58-d847-4a57-af62-16f38474609c admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:51.795084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734ab58-d847-4a57-af62-16f38474609c admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:51.795792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734ab58-d847-4a57-af62-16f38474609c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:51.796102 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734ab58-d847-4a57-af62-16f38474609c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:51.797833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734ab58-d847-4a57-af62-16f38474609c admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:51.819710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1187/2353] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:51.821634 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d289f0-65e1-4927-b5dd-6ce9e9492888 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:51.824716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b4d289f0-65e1-4927-b5dd-6ce9e9492888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:51.834284 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734ab58-d847-4a57-af62-16f38474609c admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:51.843755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d289f0-65e1-4927-b5dd-6ce9e9492888 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:51.854822 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734ab58-d847-4a57-af62-16f38474609c admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:51.862604 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b6abf-1141-4bd9-9bf5-e9eafb22f282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:51.863688 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b6abf-1141-4bd9-9bf5-e9eafb22f282 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:51.864673 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b6abf-1141-4bd9-9bf5-e9eafb22f282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:51.865009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b6abf-1141-4bd9-9bf5-e9eafb22f282 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:51.865943 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b6abf-1141-4bd9-9bf5-e9eafb22f282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:51.869753 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b06d83-9a91-4fa0-bfd2-11e31e7e0ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:51.870465 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b06d83-9a91-4fa0-bfd2-11e31e7e0ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:51.870697 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b06d83-9a91-4fa0-bfd2-11e31e7e0ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:51.871012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b06d83-9a91-4fa0-bfd2-11e31e7e0ef8 admin admin] PATH_INFO: `/v3/projects/ce36259bd4394aad8b5ae74a1234fbd6` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:51.875116 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b6abf-1141-4bd9-9bf5-e9eafb22f282 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:51.880139 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b06d83-9a91-4fa0-bfd2-11e31e7e0ef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce36259bd4394aad8b5ae74a1234fbd6)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:51.882438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b06d83-9a91-4fa0-bfd2-11e31e7e0ef8 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:51.959445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1188/2354] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:14:51 2022] POST /identity/v3/projects => generated 397 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:51.960441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b6abf-1141-4bd9-9bf5-e9eafb22f282 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:51.989896 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d2b06d83-9a91-4fa0-bfd2-11e31e7e0ef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce36259bd4394aad8b5ae74a1234fbd6', 'request_id': 'req-d2b06d83-9a91-4fa0-bfd2-11e31e7e0ef8'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:51.999082 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d2b06d83-9a91-4fa0-bfd2-11e31e7e0ef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce36259bd4394aad8b5ae74a1234fbd6', 'request_id': 'req-d2b06d83-9a91-4fa0-bfd2-11e31e7e0ef8'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:52.010325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d2b06d83-9a91-4fa0-bfd2-11e31e7e0ef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce36259bd4394aad8b5ae74a1234fbd6', 'request_id': 'req-d2b06d83-9a91-4fa0-bfd2-11e31e7e0ef8'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:52.048763 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d2b06d83-9a91-4fa0-bfd2-11e31e7e0ef8 admin admin] The token cache is being invalidate because project ce36259bd4394aad8b5ae74a1234fbd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:14:52.049459 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d2b06d83-9a91-4fa0-bfd2-11e31e7e0ef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:52.050459 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d2b06d83-9a91-4fa0-bfd2-11e31e7e0ef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:52.052775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1167/2355] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:14:51 2022] DELETE /identity/v3/projects/ce36259bd4394aad8b5ae74a1234fbd6 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:52.057008 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b06d83-9a91-4fa0-bfd2-11e31e7e0ef8 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:52.059125 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d2b06d83-9a91-4fa0-bfd2-11e31e7e0ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:52.077322 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b6abf-1141-4bd9-9bf5-e9eafb22f282 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:52.127710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:52.159443 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:52.189672 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b06d83-9a91-4fa0-bfd2-11e31e7e0ef8 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:52.199021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae3b54-2030-4a1c-a15e-353161dd3821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:52.199852 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae3b54-2030-4a1c-a15e-353161dd3821 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:52.200386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae3b54-2030-4a1c-a15e-353161dd3821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:52.200745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae3b54-2030-4a1c-a15e-353161dd3821 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:52.201061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310fdb5-b816-4ac2-991c-31f037481bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:52.201336 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae3b54-2030-4a1c-a15e-353161dd3821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:52.201787 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310fdb5-b816-4ac2-991c-31f037481bb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:52.202067 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310fdb5-b816-4ac2-991c-31f037481bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:52.202341 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310fdb5-b816-4ac2-991c-31f037481bb3 admin admin] PATH_INFO: `/v3/users/f430f6e0afbb46faaf78ff616383f7e2` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:52.204529 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae3b54-2030-4a1c-a15e-353161dd3821 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:52.205291 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310fdb5-b816-4ac2-991c-31f037481bb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f430f6e0afbb46faaf78ff616383f7e2)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:52.212008 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310fdb5-b816-4ac2-991c-31f037481bb3 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:52.237840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1189/2356] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:14:51 2022] POST /identity/v3/users => generated 299 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:52.238880 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae3b54-2030-4a1c-a15e-353161dd3821 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:52.241937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6aae3b54-2030-4a1c-a15e-353161dd3821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:52.260043 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae3b54-2030-4a1c-a15e-353161dd3821 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:52.268385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd8d7d-9b99-4f9b-b931-14c777fb291b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:52.269203 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd8d7d-9b99-4f9b-b931-14c777fb291b admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:52.269682 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd8d7d-9b99-4f9b-b931-14c777fb291b admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:52.270021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd8d7d-9b99-4f9b-b931-14c777fb291b admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:52.270740 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd8d7d-9b99-4f9b-b931-14c777fb291b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:52.273528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd8d7d-9b99-4f9b-b931-14c777fb291b admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:52.333246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1190/2357] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:52.337398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd8d7d-9b99-4f9b-b931-14c777fb291b admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:52.357460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b310fdb5-b816-4ac2-991c-31f037481bb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f430f6e0afbb46faaf78ff616383f7e2'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:52.358322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b310fdb5-b816-4ac2-991c-31f037481bb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f430f6e0afbb46faaf78ff616383f7e2'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:52.359234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b310fdb5-b816-4ac2-991c-31f037481bb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f430f6e0afbb46faaf78ff616383f7e2'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:52.365101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd8d7d-9b99-4f9b-b931-14c777fb291b admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:52.375192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b310fdb5-b816-4ac2-991c-31f037481bb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f430f6e0afbb46faaf78ff616383f7e2'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:52.375536 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fe4d3-5e7c-43d0-8dca-fb1675d65642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:52.376694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fe4d3-5e7c-43d0-8dca-fb1675d65642 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:52.377215 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fe4d3-5e7c-43d0-8dca-fb1675d65642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:52.377700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fe4d3-5e7c-43d0-8dca-fb1675d65642 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:52.378757 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fe4d3-5e7c-43d0-8dca-fb1675d65642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:52.380883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fe4d3-5e7c-43d0-8dca-fb1675d65642 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:52.401221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1191/2358] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:52.404566 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b310fdb5-b816-4ac2-991c-31f037481bb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f430f6e0afbb46faaf78ff616383f7e2'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:52.406313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fe4d3-5e7c-43d0-8dca-fb1675d65642 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:52.409272 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9f4fe4d3-5e7c-43d0-8dca-fb1675d65642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:52.423242 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1168/2359] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:14:52 2022] DELETE /identity/v3/users/f430f6e0afbb46faaf78ff616383f7e2 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:52.427228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fe4d3-5e7c-43d0-8dca-fb1675d65642 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:52.511035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede083e-097e-4891-8d16-b9dd37ba878d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:52.512376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede083e-097e-4891-8d16-b9dd37ba878d admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:52.512822 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede083e-097e-4891-8d16-b9dd37ba878d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:52.513100 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede083e-097e-4891-8d16-b9dd37ba878d admin admin] PATH_INFO: `/v3/projects/2d698779e6e84255ac335750968491f2/users/5f0977da146447bdac640e104634135e/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:52.540369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede083e-097e-4891-8d16-b9dd37ba878d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d698779e6e84255ac335750968491f2, user_id=5f0977da146447bdac640e104634135e, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:52.544581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede083e-097e-4891-8d16-b9dd37ba878d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:52.564121 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1192/2360] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:14:52 2022] PUT /identity/v3/projects/2d698779e6e84255ac335750968491f2/users/5f0977da146447bdac640e104634135e/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:52.589473 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7cf3f-74ca-4e7f-a355-8bcffe2763f1 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:52.589770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7cf3f-74ca-4e7f-a355-8bcffe2763f1 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:52.590462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7cf3f-74ca-4e7f-a355-8bcffe2763f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:52.632755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-0ff7cf3f-74ca-4e7f-a355-8bcffe2763f1 None None] MFA Rules not processed for user `5f0977da146447bdac640e104634135e`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:52.634233 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-0ff7cf3f-74ca-4e7f-a355-8bcffe2763f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:52.755969 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1169/2361] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:52 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:52.831746 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede083e-097e-4891-8d16-b9dd37ba878d admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:52.834373 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-aede083e-097e-4891-8d16-b9dd37ba878d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:52.885001 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7cf3f-74ca-4e7f-a355-8bcffe2763f1 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:52.889782 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-0ff7cf3f-74ca-4e7f-a355-8bcffe2763f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:52.962228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:52.963755 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede083e-097e-4891-8d16-b9dd37ba878d admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:52.973960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaae331-5634-492b-88ed-407cf05aa0e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:52.974810 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaae331-5634-492b-88ed-407cf05aa0e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:52.975130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaae331-5634-492b-88ed-407cf05aa0e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:52.975494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaae331-5634-492b-88ed-407cf05aa0e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:53.000766 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7cf3f-74ca-4e7f-a355-8bcffe2763f1 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:53.005749 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaae331-5634-492b-88ed-407cf05aa0e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:53.007292 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaae331-5634-492b-88ed-407cf05aa0e5 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:53.008074 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda437a-07c2-41aa-9ed1-57d1291ebdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:53.009116 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda437a-07c2-41aa-9ed1-57d1291ebdf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:53.009440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda437a-07c2-41aa-9ed1-57d1291ebdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:53.009720 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda437a-07c2-41aa-9ed1-57d1291ebdf8 admin admin] PATH_INFO: `/v3/projects/a15a28638b4b49e382ab0cb9f3175fd5` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:53.011024 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda437a-07c2-41aa-9ed1-57d1291ebdf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a15a28638b4b49e382ab0cb9f3175fd5)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:53.016064 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda437a-07c2-41aa-9ed1-57d1291ebdf8 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:53.021565 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1193/2362] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:52 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:14:53.087144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-6bda437a-07c2-41aa-9ed1-57d1291ebdf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a15a28638b4b49e382ab0cb9f3175fd5', 'request_id': 'req-6bda437a-07c2-41aa-9ed1-57d1291ebdf8'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:53.093891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-6bda437a-07c2-41aa-9ed1-57d1291ebdf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a15a28638b4b49e382ab0cb9f3175fd5', 'request_id': 'req-6bda437a-07c2-41aa-9ed1-57d1291ebdf8'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:53.105719 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-6bda437a-07c2-41aa-9ed1-57d1291ebdf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a15a28638b4b49e382ab0cb9f3175fd5', 'request_id': 'req-6bda437a-07c2-41aa-9ed1-57d1291ebdf8'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:53.118971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-6bda437a-07c2-41aa-9ed1-57d1291ebdf8 admin admin] The token cache is being invalidate because project a15a28638b4b49e382ab0cb9f3175fd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:14:53.119284 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-6bda437a-07c2-41aa-9ed1-57d1291ebdf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:53.120436 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-6bda437a-07c2-41aa-9ed1-57d1291ebdf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:53.121999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1170/2363] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:14:52 2022] DELETE /identity/v3/projects/a15a28638b4b49e382ab0cb9f3175fd5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:14:53.127234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaae331-5634-492b-88ed-407cf05aa0e5 service cinder] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:53.155185 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaae331-5634-492b-88ed-407cf05aa0e5 service cinder] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:53.210199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:53.324102 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0763f6-676a-4b22-b146-76d9be564840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:53.324102 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0763f6-676a-4b22-b146-76d9be564840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:53.324102 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0763f6-676a-4b22-b146-76d9be564840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:53.324102 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0763f6-676a-4b22-b146-76d9be564840 admin admin] PATH_INFO: `/v3/users/a18ba39d160e4ba3ad83d0a8002028c4` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:53.324659 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0763f6-676a-4b22-b146-76d9be564840 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a18ba39d160e4ba3ad83d0a8002028c4)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:53.324659 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0763f6-676a-4b22-b146-76d9be564840 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:53.405840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-4e0763f6-676a-4b22-b146-76d9be564840 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a18ba39d160e4ba3ad83d0a8002028c4'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:53.406531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-4e0763f6-676a-4b22-b146-76d9be564840 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a18ba39d160e4ba3ad83d0a8002028c4'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:53.407148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-4e0763f6-676a-4b22-b146-76d9be564840 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a18ba39d160e4ba3ad83d0a8002028c4'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:53.414792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-4e0763f6-676a-4b22-b146-76d9be564840 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a18ba39d160e4ba3ad83d0a8002028c4'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:53.430791 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-4e0763f6-676a-4b22-b146-76d9be564840 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a18ba39d160e4ba3ad83d0a8002028c4'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:53.449560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1194/2364] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:14:53 2022] DELETE /identity/v3/users/a18ba39d160e4ba3ad83d0a8002028c4 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:53.851577 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda437a-07c2-41aa-9ed1-57d1291ebdf8 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:53.853310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-6bda437a-07c2-41aa-9ed1-57d1291ebdf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:53.867337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda437a-07c2-41aa-9ed1-57d1291ebdf8 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:53.920938 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0763f6-676a-4b22-b146-76d9be564840 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:53.923004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4e0763f6-676a-4b22-b146-76d9be564840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:53.942315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1fe4f9-571f-40fd-9a02-73b82beca186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:53.942996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fe4f9-571f-40fd-9a02-73b82beca186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:53.943266 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fe4f9-571f-40fd-9a02-73b82beca186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:53.945352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fe4f9-571f-40fd-9a02-73b82beca186 admin admin] PATH_INFO: `/v3/users/b016f4b3e0ce4197a63d139e72c1856b` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:53.946759 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1fe4f9-571f-40fd-9a02-73b82beca186 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b016f4b3e0ce4197a63d139e72c1856b)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:53.949007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1fe4f9-571f-40fd-9a02-73b82beca186 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:54.012114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-3a1fe4f9-571f-40fd-9a02-73b82beca186 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b016f4b3e0ce4197a63d139e72c1856b'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:54.012614 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0763f6-676a-4b22-b146-76d9be564840 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:54.013498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-3a1fe4f9-571f-40fd-9a02-73b82beca186 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b016f4b3e0ce4197a63d139e72c1856b'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:54.014335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-3a1fe4f9-571f-40fd-9a02-73b82beca186 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b016f4b3e0ce4197a63d139e72c1856b'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:54.017731 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31328b9f-c737-4379-a3dd-5b1873463625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:54.018491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31328b9f-c737-4379-a3dd-5b1873463625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:54.018839 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31328b9f-c737-4379-a3dd-5b1873463625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:54.019213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31328b9f-c737-4379-a3dd-5b1873463625 admin admin] PATH_INFO: `/v3/projects/6f51fdc6654c4332b38f7589dea95f70` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:54.020472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31328b9f-c737-4379-a3dd-5b1873463625 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f51fdc6654c4332b38f7589dea95f70)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:54.022482 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31328b9f-c737-4379-a3dd-5b1873463625 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:54.023640 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-3a1fe4f9-571f-40fd-9a02-73b82beca186 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b016f4b3e0ce4197a63d139e72c1856b'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:54.037094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-3a1fe4f9-571f-40fd-9a02-73b82beca186 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b016f4b3e0ce4197a63d139e72c1856b'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:54.041632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:54.063254 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1171/2365] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:14:53 2022] DELETE /identity/v3/users/b016f4b3e0ce4197a63d139e72c1856b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:54.068498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a99c52-c958-4298-8fae-45bcf351c025 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:54.068853 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a99c52-c958-4298-8fae-45bcf351c025 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:54.069197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a99c52-c958-4298-8fae-45bcf351c025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:54.080719 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-31328b9f-c737-4379-a3dd-5b1873463625 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f51fdc6654c4332b38f7589dea95f70', 'request_id': 'req-31328b9f-c737-4379-a3dd-5b1873463625'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:54.081851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:54.092160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-31328b9f-c737-4379-a3dd-5b1873463625 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f51fdc6654c4332b38f7589dea95f70', 'request_id': 'req-31328b9f-c737-4379-a3dd-5b1873463625'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:54.101872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-d7a99c52-c958-4298-8fae-45bcf351c025 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:54.103296 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d7a99c52-c958-4298-8fae-45bcf351c025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:54.104701 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-31328b9f-c737-4379-a3dd-5b1873463625 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f51fdc6654c4332b38f7589dea95f70', 'request_id': 'req-31328b9f-c737-4379-a3dd-5b1873463625'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:54.119446 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-31328b9f-c737-4379-a3dd-5b1873463625 admin admin] The token cache is being invalidate because project 6f51fdc6654c4332b38f7589dea95f70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:14:54.120020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-31328b9f-c737-4379-a3dd-5b1873463625 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:54.120600 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-31328b9f-c737-4379-a3dd-5b1873463625 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:54.122667 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1195/2366] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:14:53 2022] DELETE /identity/v3/projects/6f51fdc6654c4332b38f7589dea95f70 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:54.128689 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024dd0c-c4f3-4af7-8395-2fdae27d44f0 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:54.128974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024dd0c-c4f3-4af7-8395-2fdae27d44f0 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:54.129350 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024dd0c-c4f3-4af7-8395-2fdae27d44f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:54.150911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-6024dd0c-c4f3-4af7-8395-2fdae27d44f0 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:54.152328 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6024dd0c-c4f3-4af7-8395-2fdae27d44f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:54.179544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1172/2367] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:54.191958 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a99c52-c958-4298-8fae-45bcf351c025 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:54.193410 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d7a99c52-c958-4298-8fae-45bcf351c025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:54.239612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1196/2368] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:14:54.243031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441c5a6-7a00-42ec-8be6-de871aaf9e5b None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:54.243481 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441c5a6-7a00-42ec-8be6-de871aaf9e5b None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:54.243828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441c5a6-7a00-42ec-8be6-de871aaf9e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:54.265134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a99c52-c958-4298-8fae-45bcf351c025 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:54.265784 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-a441c5a6-7a00-42ec-8be6-de871aaf9e5b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:54.266975 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a441c5a6-7a00-42ec-8be6-de871aaf9e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:54.269359 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e93c0-d72d-483d-9065-ba41bf91d406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:54.270020 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e93c0-d72d-483d-9065-ba41bf91d406 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:54.270299 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e93c0-d72d-483d-9065-ba41bf91d406 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:54.270601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e93c0-d72d-483d-9065-ba41bf91d406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:54.275329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1197/2369] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:54.280101 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441c5a6-7a00-42ec-8be6-de871aaf9e5b None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:54.282248 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e93c0-d72d-483d-9065-ba41bf91d406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:54.284927 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e93c0-d72d-483d-9065-ba41bf91d406 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:54.296489 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441c5a6-7a00-42ec-8be6-de871aaf9e5b None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:54.299023 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1173/2370] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:14:54.302047 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a27ea1-31be-4351-97a7-550a2251165e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:54.302730 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a27ea1-31be-4351-97a7-550a2251165e admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:54.303006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a27ea1-31be-4351-97a7-550a2251165e admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:54.303283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a27ea1-31be-4351-97a7-550a2251165e admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:54.304131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a27ea1-31be-4351-97a7-550a2251165e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:54.304445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a27ea1-31be-4351-97a7-550a2251165e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:54.306246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a27ea1-31be-4351-97a7-550a2251165e admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:54.322512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1198/2371] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:14:54.324177 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e93c0-d72d-483d-9065-ba41bf91d406 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:54.335485 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e93c0-d72d-483d-9065-ba41bf91d406 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:54.341069 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c3e98-5ebc-4d4d-ab44-ecc86d062ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:54.341714 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c3e98-5ebc-4d4d-ab44-ecc86d062ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:54.341988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c3e98-5ebc-4d4d-ab44-ecc86d062ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:54.342253 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c3e98-5ebc-4d4d-ab44-ecc86d062ae3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:54.342953 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c3e98-5ebc-4d4d-ab44-ecc86d062ae3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:54.344856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c3e98-5ebc-4d4d-ab44-ecc86d062ae3 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:54.398350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1174/2372] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:14:54 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:54.403842 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a27ea1-31be-4351-97a7-550a2251165e admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:54.426411 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a27ea1-31be-4351-97a7-550a2251165e admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:54.528355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32634e2-c796-49c3-b30e-60c796afccfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:54.528970 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32634e2-c796-49c3-b30e-60c796afccfc admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:54.529249 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32634e2-c796-49c3-b30e-60c796afccfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:54.529528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32634e2-c796-49c3-b30e-60c796afccfc admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:54.530171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32634e2-c796-49c3-b30e-60c796afccfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:54.532149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32634e2-c796-49c3-b30e-60c796afccfc admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:54.557535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1199/2373] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:14:54 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:14:54.564590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c3e98-5ebc-4d4d-ab44-ecc86d062ae3 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:54.577205 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c3e98-5ebc-4d4d-ab44-ecc86d062ae3 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:54.583314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2aa492-13f2-42ea-85e9-41cb91258e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:54.583973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2aa492-13f2-42ea-85e9-41cb91258e04 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:54.584250 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2aa492-13f2-42ea-85e9-41cb91258e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:54.584522 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2aa492-13f2-42ea-85e9-41cb91258e04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:54.585170 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2aa492-13f2-42ea-85e9-41cb91258e04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:54.586812 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2aa492-13f2-42ea-85e9-41cb91258e04 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:54.600613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1175/2374] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:54.604717 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32634e2-c796-49c3-b30e-60c796afccfc admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:54.614897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32634e2-c796-49c3-b30e-60c796afccfc admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:54.619792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87073f6-46ad-45cd-a617-4ad82fbd0c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:54.620451 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87073f6-46ad-45cd-a617-4ad82fbd0c3b admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:54.620761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87073f6-46ad-45cd-a617-4ad82fbd0c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:54.621035 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87073f6-46ad-45cd-a617-4ad82fbd0c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:54.621743 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87073f6-46ad-45cd-a617-4ad82fbd0c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:54.623368 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87073f6-46ad-45cd-a617-4ad82fbd0c3b admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:54.634712 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1200/2375] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:54.639025 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2aa492-13f2-42ea-85e9-41cb91258e04 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:54.648986 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2aa492-13f2-42ea-85e9-41cb91258e04 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:54.655310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6fbbb-c9c0-42c2-83be-3c0b95d19a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:54.656040 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6fbbb-c9c0-42c2-83be-3c0b95d19a2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:54.656212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6fbbb-c9c0-42c2-83be-3c0b95d19a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:54.656481 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6fbbb-c9c0-42c2-83be-3c0b95d19a2c admin admin] PATH_INFO: `/v3/projects/6e209b5695624d0e84f93d375c7fe1ab/users/bab4b09414344ef2b2088cf96378f21b/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:54.672926 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6fbbb-c9c0-42c2-83be-3c0b95d19a2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e209b5695624d0e84f93d375c7fe1ab, user_id=bab4b09414344ef2b2088cf96378f21b, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:54.675768 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6fbbb-c9c0-42c2-83be-3c0b95d19a2c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:54.700879 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1176/2376] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:14:54 2022] PUT /identity/v3/projects/6e209b5695624d0e84f93d375c7fe1ab/users/bab4b09414344ef2b2088cf96378f21b/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:54.724351 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6fbbb-c9c0-42c2-83be-3c0b95d19a2c admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:54.724840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51b3b1-c0e9-4f14-a574-b698ba89c751 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:54.725132 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51b3b1-c0e9-4f14-a574-b698ba89c751 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:54.726234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3fb6fbbb-c9c0-42c2-83be-3c0b95d19a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:54.727180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51b3b1-c0e9-4f14-a574-b698ba89c751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:54.784976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:54.808067 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-0f51b3b1-c0e9-4f14-a574-b698ba89c751 None None] MFA Rules not processed for user `bab4b09414344ef2b2088cf96378f21b`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:54.812680 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-0f51b3b1-c0e9-4f14-a574-b698ba89c751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:54.818237 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6fbbb-c9c0-42c2-83be-3c0b95d19a2c admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:54.827548 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158eda37-32b4-411d-95fc-167119ea24d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:54.828239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158eda37-32b4-411d-95fc-167119ea24d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:54.828517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158eda37-32b4-411d-95fc-167119ea24d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:54.828787 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158eda37-32b4-411d-95fc-167119ea24d9 admin admin] PATH_INFO: `/v3/projects/711ca60ec25e46e3acd135dac97476f5` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:54.829952 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158eda37-32b4-411d-95fc-167119ea24d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=711ca60ec25e46e3acd135dac97476f5)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:54.831927 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158eda37-32b4-411d-95fc-167119ea24d9 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:54.882069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:54.927835 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-158eda37-32b4-411d-95fc-167119ea24d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '711ca60ec25e46e3acd135dac97476f5', 'request_id': 'req-158eda37-32b4-411d-95fc-167119ea24d9'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:54.945151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-158eda37-32b4-411d-95fc-167119ea24d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '711ca60ec25e46e3acd135dac97476f5', 'request_id': 'req-158eda37-32b4-411d-95fc-167119ea24d9'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:54.963950 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-158eda37-32b4-411d-95fc-167119ea24d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '711ca60ec25e46e3acd135dac97476f5', 'request_id': 'req-158eda37-32b4-411d-95fc-167119ea24d9'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:54.971475 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1201/2377] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:54.985766 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51b3b1-c0e9-4f14-a574-b698ba89c751 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:54.987406 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-0f51b3b1-c0e9-4f14-a574-b698ba89c751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:54.992648 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-158eda37-32b4-411d-95fc-167119ea24d9 admin admin] The token cache is being invalidate because project 711ca60ec25e46e3acd135dac97476f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:14:54.995367 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-158eda37-32b4-411d-95fc-167119ea24d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:55.002341 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-158eda37-32b4-411d-95fc-167119ea24d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:14:55.010311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1177/2378] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:14:54 2022] DELETE /identity/v3/projects/711ca60ec25e46e3acd135dac97476f5 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:55.015449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa97153-f761-4c77-8b6c-163876adf599 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:55.016133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa97153-f761-4c77-8b6c-163876adf599 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:55.016133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa97153-f761-4c77-8b6c-163876adf599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:55.067887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-3aa97153-f761-4c77-8b6c-163876adf599 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:55.072041 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3aa97153-f761-4c77-8b6c-163876adf599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:55.146418 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51b3b1-c0e9-4f14-a574-b698ba89c751 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:55.220612 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1178/2379] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:55.230143 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbfdcb-d477-4acd-a164-67b5c0f4bcfe None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:55.230456 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbfdcb-d477-4acd-a164-67b5c0f4bcfe None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:55.230984 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbfdcb-d477-4acd-a164-67b5c0f4bcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:55.262513 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-adcbfdcb-d477-4acd-a164-67b5c0f4bcfe None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:55.268064 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-adcbfdcb-d477-4acd-a164-67b5c0f4bcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:55.280476 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261067d6-e424-4537-bc2f-76883454e47c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:55.281260 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261067d6-e424-4537-bc2f-76883454e47c service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:55.281607 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261067d6-e424-4537-bc2f-76883454e47c service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:55.281953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261067d6-e424-4537-bc2f-76883454e47c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:55.283978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-261067d6-e424-4537-bc2f-76883454e47c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:55.302781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1179/2380] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:55.308405 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbfdcb-d477-4acd-a164-67b5c0f4bcfe None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:55.329908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbfdcb-d477-4acd-a164-67b5c0f4bcfe None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:55.336579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f5e0f-6e5d-4486-ba97-f0d7e0b81b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:55.337219 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f5e0f-6e5d-4486-ba97-f0d7e0b81b2b admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:55.337503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f5e0f-6e5d-4486-ba97-f0d7e0b81b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:55.337791 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f5e0f-6e5d-4486-ba97-f0d7e0b81b2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:55.339545 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f5e0f-6e5d-4486-ba97-f0d7e0b81b2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:14:55.339891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f5e0f-6e5d-4486-ba97-f0d7e0b81b2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:55.341765 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f5e0f-6e5d-4486-ba97-f0d7e0b81b2b admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:55.364707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1180/2381] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:14:55.369243 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f5e0f-6e5d-4486-ba97-f0d7e0b81b2b admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:55.383585 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f5e0f-6e5d-4486-ba97-f0d7e0b81b2b admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:55.388968 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8e82d-7908-4791-9942-6517787bdccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:55.389603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8e82d-7908-4791-9942-6517787bdccc admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:55.389885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8e82d-7908-4791-9942-6517787bdccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:55.390160 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8e82d-7908-4791-9942-6517787bdccc admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:55.390809 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8e82d-7908-4791-9942-6517787bdccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:55.393974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8e82d-7908-4791-9942-6517787bdccc admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:55.395017 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261067d6-e424-4537-bc2f-76883454e47c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:55.396774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261067d6-e424-4537-bc2f-76883454e47c service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:55.409988 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1202/2382] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:14:55.429718 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1181/2383] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:14:55 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:14:55.430949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261067d6-e424-4537-bc2f-76883454e47c service cinder] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:55.445067 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261067d6-e424-4537-bc2f-76883454e47c service cinder] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:55.499868 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:55.534105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ca13a-77f1-491a-94f5-bda17d8185a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:55.535119 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ca13a-77f1-491a-94f5-bda17d8185a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:55.535510 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ca13a-77f1-491a-94f5-bda17d8185a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:55.535859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ca13a-77f1-491a-94f5-bda17d8185a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:55.536796 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ca13a-77f1-491a-94f5-bda17d8185a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:55.541024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ca13a-77f1-491a-94f5-bda17d8185a1 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:55.573959 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1203/2384] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:14:55 2022] POST /identity/v3/users => generated 286 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:14:55.574949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8e82d-7908-4791-9942-6517787bdccc admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:55.587745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8e82d-7908-4791-9942-6517787bdccc admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:55.594051 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775fa17-3258-4c05-b01f-2edaa7b659b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:55.594906 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775fa17-3258-4c05-b01f-2edaa7b659b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:55.595581 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775fa17-3258-4c05-b01f-2edaa7b659b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:55.596069 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775fa17-3258-4c05-b01f-2edaa7b659b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:55.598410 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775fa17-3258-4c05-b01f-2edaa7b659b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:55.600801 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775fa17-3258-4c05-b01f-2edaa7b659b3 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:55.616017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1182/2385] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:14:55.619860 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ca13a-77f1-491a-94f5-bda17d8185a1 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:55.638348 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ca13a-77f1-491a-94f5-bda17d8185a1 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:55.646119 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11db7b7-087c-4780-a8ba-ac5c93035b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:55.646742 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11db7b7-087c-4780-a8ba-ac5c93035b38 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:55.647135 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11db7b7-087c-4780-a8ba-ac5c93035b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:55.647298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11db7b7-087c-4780-a8ba-ac5c93035b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:55.647987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11db7b7-087c-4780-a8ba-ac5c93035b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:55.650344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11db7b7-087c-4780-a8ba-ac5c93035b38 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:55.667820 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1204/2386] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:14:55.673086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775fa17-3258-4c05-b01f-2edaa7b659b3 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:55.686126 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775fa17-3258-4c05-b01f-2edaa7b659b3 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:55.692072 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02b190-4088-47e8-88a5-099ad0f06496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:55.692745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02b190-4088-47e8-88a5-099ad0f06496 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:55.693023 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02b190-4088-47e8-88a5-099ad0f06496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:55.693300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02b190-4088-47e8-88a5-099ad0f06496 admin admin] PATH_INFO: `/v3/projects/fd004facbaed41a7a54a3ed6a6a44305/users/c26232f042844a898c19d1f63152b428/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:55.715557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02b190-4088-47e8-88a5-099ad0f06496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd004facbaed41a7a54a3ed6a6a44305, user_id=c26232f042844a898c19d1f63152b428, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:55.719724 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02b190-4088-47e8-88a5-099ad0f06496 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:55.723401 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:14:55.741395 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1183/2387] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:14:55 2022] PUT /identity/v3/projects/fd004facbaed41a7a54a3ed6a6a44305/users/c26232f042844a898c19d1f63152b428/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:14:55.747684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a856a66-d13b-4f76-93fe-94c4bf273400 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:55.748039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a856a66-d13b-4f76-93fe-94c4bf273400 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:55.748361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a856a66-d13b-4f76-93fe-94c4bf273400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:55.752991 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb3fd5-6579-4dc8-bfcc-1877d5f51099 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:55.753279 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb3fd5-6579-4dc8-bfcc-1877d5f51099 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:55.753554 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb3fd5-6579-4dc8-bfcc-1877d5f51099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:55.774870 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-19cb3fd5-6579-4dc8-bfcc-1877d5f51099 None None] MFA Rules not processed for user `b3f1ae88e29f4172a7db7393d517a893`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:55.776156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-19cb3fd5-6579-4dc8-bfcc-1877d5f51099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:55.786217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-8a856a66-d13b-4f76-93fe-94c4bf273400 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:55.787647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8a856a66-d13b-4f76-93fe-94c4bf273400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:55.873723 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1184/2388] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 23 07:14:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:55.891353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1205/2389] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:55.903489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb3fd5-6579-4dc8-bfcc-1877d5f51099 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:55.905522 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-19cb3fd5-6579-4dc8-bfcc-1877d5f51099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:55.943126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a856a66-d13b-4f76-93fe-94c4bf273400 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:55.944665 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8a856a66-d13b-4f76-93fe-94c4bf273400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:56.009773 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb3fd5-6579-4dc8-bfcc-1877d5f51099 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:56.014537 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e474d4-5733-4fa1-9796-90520cbaa556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:56.015236 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e474d4-5733-4fa1-9796-90520cbaa556 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:56.015613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e474d4-5733-4fa1-9796-90520cbaa556 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:56.015949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e474d4-5733-4fa1-9796-90520cbaa556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:56.027172 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e474d4-5733-4fa1-9796-90520cbaa556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:56.029079 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e474d4-5733-4fa1-9796-90520cbaa556 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:56.039609 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1185/2390] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:14:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:14:56.076963 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a856a66-d13b-4f76-93fe-94c4bf273400 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:56.084503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4bc1c-68a3-45d3-be76-2e2eff56b3f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd88ec8b477443af95bff9af03471adf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:56.085062 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4bc1c-68a3-45d3-be76-2e2eff56b3f5 service swift] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:56.085466 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4bc1c-68a3-45d3-be76-2e2eff56b3f5 service swift] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:56.085879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4bc1c-68a3-45d3-be76-2e2eff56b3f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:56.100375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4bc1c-68a3-45d3-be76-2e2eff56b3f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:56.100979 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4bc1c-68a3-45d3-be76-2e2eff56b3f5 service swift] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:56.110856 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1206/2391] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 23 07:14:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 07:14:56.805110 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60124c-c2b9-4e08-9bf9-0b23e1e8c91a None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:56.805476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60124c-c2b9-4e08-9bf9-0b23e1e8c91a None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:56.805822 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60124c-c2b9-4e08-9bf9-0b23e1e8c91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:56.826427 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-ae60124c-c2b9-4e08-9bf9-0b23e1e8c91a None None] MFA Rules not processed for user `b3f1ae88e29f4172a7db7393d517a893`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:56.828915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ae60124c-c2b9-4e08-9bf9-0b23e1e8c91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:56.837085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1186/2392] 10.222.0.17 () {58 vars in 967 bytes} [Tue Aug 23 07:14:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:56.853433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4bc1c-68a3-45d3-be76-2e2eff56b3f5 service swift] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:56.865110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4bc1c-68a3-45d3-be76-2e2eff56b3f5 service swift] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:56.871951 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c10315-a041-492b-8c6d-4fa5e781caa6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd88ec8b477443af95bff9af03471adf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:56.872895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c10315-a041-492b-8c6d-4fa5e781caa6 service swift] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:56.873397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c10315-a041-492b-8c6d-4fa5e781caa6 service swift] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:56.873888 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c10315-a041-492b-8c6d-4fa5e781caa6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:56.883839 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c10315-a041-492b-8c6d-4fa5e781caa6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:56.885818 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c10315-a041-492b-8c6d-4fa5e781caa6 service swift] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:56.896512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1207/2393] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 23 07:14:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 07:14:57.182883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc9641-057e-4159-983d-3f90da0a6921 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:57.183368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc9641-057e-4159-983d-3f90da0a6921 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:57.183467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc9641-057e-4159-983d-3f90da0a6921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:57.203324 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-9dcc9641-057e-4159-983d-3f90da0a6921 None None] MFA Rules not processed for user `b3f1ae88e29f4172a7db7393d517a893`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:57.204587 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9dcc9641-057e-4159-983d-3f90da0a6921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:57.212384 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1187/2394] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 23 07:14:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:57.225618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c10315-a041-492b-8c6d-4fa5e781caa6 service swift] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:57.237120 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c10315-a041-492b-8c6d-4fa5e781caa6 service swift] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:57.242829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee9b24-d29b-4dc6-be1c-ce28250a6926 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd88ec8b477443af95bff9af03471adf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:57.243726 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee9b24-d29b-4dc6-be1c-ce28250a6926 service swift] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:57.244002 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee9b24-d29b-4dc6-be1c-ce28250a6926 service swift] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:57.244270 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee9b24-d29b-4dc6-be1c-ce28250a6926 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:57.254175 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee9b24-d29b-4dc6-be1c-ce28250a6926 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:57.256115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee9b24-d29b-4dc6-be1c-ce28250a6926 service swift] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:57.267248 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1208/2395] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 23 07:14:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 07:14:57.729361 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b5fe8-9c87-4252-af54-ad28cf12cf5b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:57.729677 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b5fe8-9c87-4252-af54-ad28cf12cf5b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:57.730116 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b5fe8-9c87-4252-af54-ad28cf12cf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:57.752290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-335b5fe8-9c87-4252-af54-ad28cf12cf5b None None] MFA Rules not processed for user `b3f1ae88e29f4172a7db7393d517a893`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:14:57.753770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-335b5fe8-9c87-4252-af54-ad28cf12cf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:14:57.763709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1188/2396] 10.222.0.17 () {58 vars in 967 bytes} [Tue Aug 23 07:14:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:14:57.784607 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee9b24-d29b-4dc6-be1c-ce28250a6926 service swift] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:14:57.799100 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee9b24-d29b-4dc6-be1c-ce28250a6926 service swift] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:14:57.807315 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad3a4b9-ac07-4fb8-a583-11d986343a19 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd88ec8b477443af95bff9af03471adf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:14:57.808107 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad3a4b9-ac07-4fb8-a583-11d986343a19 service swift] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:14:57.808491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad3a4b9-ac07-4fb8-a583-11d986343a19 service swift] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:14:57.809007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad3a4b9-ac07-4fb8-a583-11d986343a19 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:14:57.837550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad3a4b9-ac07-4fb8-a583-11d986343a19 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:14:57.840697 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad3a4b9-ac07-4fb8-a583-11d986343a19 service swift] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:14:57.852363 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1209/2397] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 23 07:14:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 07:15:00.925477 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b5fe8-9c87-4252-af54-ad28cf12cf5b None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:00.927335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-335b5fe8-9c87-4252-af54-ad28cf12cf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:00.943621 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b5fe8-9c87-4252-af54-ad28cf12cf5b None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:00.950814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c42e3-33d0-40fb-bb8b-c5453cd42114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:00.951608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c42e3-33d0-40fb-bb8b-c5453cd42114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:00.951961 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c42e3-33d0-40fb-bb8b-c5453cd42114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:00.952305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c42e3-33d0-40fb-bb8b-c5453cd42114 admin admin] PATH_INFO: `/v3/users/6ed299381d5b49fb8577287af4368fe1` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:00.953549 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c42e3-33d0-40fb-bb8b-c5453cd42114 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ed299381d5b49fb8577287af4368fe1)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:00.955795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c42e3-33d0-40fb-bb8b-c5453cd42114 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:01.035815 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad3a4b9-ac07-4fb8-a583-11d986343a19 service swift] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:01.038686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6ad3a4b9-ac07-4fb8-a583-11d986343a19 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:01.075862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d92c42e3-33d0-40fb-bb8b-c5453cd42114 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ed299381d5b49fb8577287af4368fe1'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:01.076594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d92c42e3-33d0-40fb-bb8b-c5453cd42114 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ed299381d5b49fb8577287af4368fe1'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:01.077326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d92c42e3-33d0-40fb-bb8b-c5453cd42114 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ed299381d5b49fb8577287af4368fe1'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:01.085816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad3a4b9-ac07-4fb8-a583-11d986343a19 service swift] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:01.102342 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81659b8c-1a46-4090-bca2-7050adf7f59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:01.102608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d92c42e3-33d0-40fb-bb8b-c5453cd42114 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ed299381d5b49fb8577287af4368fe1'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:01.103590 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81659b8c-1a46-4090-bca2-7050adf7f59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:01.105235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81659b8c-1a46-4090-bca2-7050adf7f59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:01.106045 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81659b8c-1a46-4090-bca2-7050adf7f59d admin admin] PATH_INFO: `/v3/users/5f0977da146447bdac640e104634135e` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:01.107748 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81659b8c-1a46-4090-bca2-7050adf7f59d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f0977da146447bdac640e104634135e)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:01.113660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81659b8c-1a46-4090-bca2-7050adf7f59d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:01.116072 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:01.135194 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d92c42e3-33d0-40fb-bb8b-c5453cd42114 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ed299381d5b49fb8577287af4368fe1'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:01.140645 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:01.179000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1189/2398] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:15:00 2022] DELETE /identity/v3/users/6ed299381d5b49fb8577287af4368fe1 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:01.180472 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6f9f7-3503-4f5b-9a96-390ab33123b4 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:01.180761 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6f9f7-3503-4f5b-9a96-390ab33123b4 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:01.181109 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6f9f7-3503-4f5b-9a96-390ab33123b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:01.201467 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-81659b8c-1a46-4090-bca2-7050adf7f59d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f0977da146447bdac640e104634135e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:01.202233 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-81659b8c-1a46-4090-bca2-7050adf7f59d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f0977da146447bdac640e104634135e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:01.202842 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-81659b8c-1a46-4090-bca2-7050adf7f59d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f0977da146447bdac640e104634135e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:01.212165 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-81659b8c-1a46-4090-bca2-7050adf7f59d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f0977da146447bdac640e104634135e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:01.216867 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-7ad6f9f7-3503-4f5b-9a96-390ab33123b4 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:01.218009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-7ad6f9f7-3503-4f5b-9a96-390ab33123b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:01.235305 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-81659b8c-1a46-4090-bca2-7050adf7f59d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f0977da146447bdac640e104634135e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:01.265924 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1210/2399] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:15:01 2022] DELETE /identity/v3/users/5f0977da146447bdac640e104634135e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:01.270266 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a120e-3c0e-477e-a9bd-fb8adde5ce6f None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:01.270552 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a120e-3c0e-477e-a9bd-fb8adde5ce6f None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:01.270825 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a120e-3c0e-477e-a9bd-fb8adde5ce6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:01.320646 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-7c5a120e-3c0e-477e-a9bd-fb8adde5ce6f None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:01.325163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-7c5a120e-3c0e-477e-a9bd-fb8adde5ce6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:01.368793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1190/2400] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:01.378650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6f9f7-3503-4f5b-9a96-390ab33123b4 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:01.380140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-7ad6f9f7-3503-4f5b-9a96-390ab33123b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:01.473892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1211/2401] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:01.478520 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6f9f7-3503-4f5b-9a96-390ab33123b4 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:01.497503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a120e-3c0e-477e-a9bd-fb8adde5ce6f None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:01.554109 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ade4f0-f78b-49e2-8304-35a683005c8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:01.554752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ade4f0-f78b-49e2-8304-35a683005c8a service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:01.555469 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ade4f0-f78b-49e2-8304-35a683005c8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:01.555755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ade4f0-f78b-49e2-8304-35a683005c8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:01.581649 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ade4f0-f78b-49e2-8304-35a683005c8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:01.584054 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ade4f0-f78b-49e2-8304-35a683005c8a service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:01.597601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1191/2402] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:15:01.623987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a120e-3c0e-477e-a9bd-fb8adde5ce6f None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:01.644290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b313f3-5379-4bd7-b520-ea9981df5e7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:01.645106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b313f3-5379-4bd7-b520-ea9981df5e7f service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:01.646187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b313f3-5379-4bd7-b520-ea9981df5e7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:01.646563 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b313f3-5379-4bd7-b520-ea9981df5e7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:01.665680 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b313f3-5379-4bd7-b520-ea9981df5e7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:01.670962 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b313f3-5379-4bd7-b520-ea9981df5e7f service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:01.703369 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1212/2403] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:15:02.056837 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ade4f0-f78b-49e2-8304-35a683005c8a service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:02.059308 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b8ade4f0-f78b-49e2-8304-35a683005c8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:02.105444 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ade4f0-f78b-49e2-8304-35a683005c8a service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:02.112947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513b597-3958-49bf-adbb-b60f6abbcfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:02.113863 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513b597-3958-49bf-adbb-b60f6abbcfa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:02.114231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513b597-3958-49bf-adbb-b60f6abbcfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:02.114615 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513b597-3958-49bf-adbb-b60f6abbcfa0 admin admin] PATH_INFO: `/v3/projects/3d91979a6ce9468c990b9e08e267d329` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:02.115966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513b597-3958-49bf-adbb-b60f6abbcfa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d91979a6ce9468c990b9e08e267d329)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:02.118156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513b597-3958-49bf-adbb-b60f6abbcfa0 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:02.273988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:DELETE=5,keystone:SELECT=95 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:02.276760 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f513b597-3958-49bf-adbb-b60f6abbcfa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d91979a6ce9468c990b9e08e267d329', 'request_id': 'req-f513b597-3958-49bf-adbb-b60f6abbcfa0'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:02.297532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f513b597-3958-49bf-adbb-b60f6abbcfa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d91979a6ce9468c990b9e08e267d329', 'request_id': 'req-f513b597-3958-49bf-adbb-b60f6abbcfa0'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:02.317347 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f513b597-3958-49bf-adbb-b60f6abbcfa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d91979a6ce9468c990b9e08e267d329', 'request_id': 'req-f513b597-3958-49bf-adbb-b60f6abbcfa0'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:02.367496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f513b597-3958-49bf-adbb-b60f6abbcfa0 admin admin] The token cache is being invalidate because project 3d91979a6ce9468c990b9e08e267d329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:15:02.367984 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f513b597-3958-49bf-adbb-b60f6abbcfa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:02.368916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f513b597-3958-49bf-adbb-b60f6abbcfa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:02.377028 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1192/2404] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:15:02 2022] DELETE /identity/v3/projects/3d91979a6ce9468c990b9e08e267d329 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:02.378873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496813c-0a23-46c7-9f87-4ea248056485 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:02.379892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496813c-0a23-46c7-9f87-4ea248056485 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:02.380402 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496813c-0a23-46c7-9f87-4ea248056485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:02.418216 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513b597-3958-49bf-adbb-b60f6abbcfa0 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:02.419056 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-7496813c-0a23-46c7-9f87-4ea248056485 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:02.420818 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f513b597-3958-49bf-adbb-b60f6abbcfa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:02.420963 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-7496813c-0a23-46c7-9f87-4ea248056485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:02.461198 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:02.553004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1213/2405] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:02.554844 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a050aca-6974-4885-89d1-05b0e1b0761c None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:02.555143 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a050aca-6974-4885-89d1-05b0e1b0761c None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:02.555482 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a050aca-6974-4885-89d1-05b0e1b0761c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:02.592588 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-1a050aca-6974-4885-89d1-05b0e1b0761c None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:02.595632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513b597-3958-49bf-adbb-b60f6abbcfa0 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:02.596842 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-1a050aca-6974-4885-89d1-05b0e1b0761c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:02.616551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0121e5-fc0e-4b7e-a4db-260f0718e451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:02.617221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0121e5-fc0e-4b7e-a4db-260f0718e451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:02.617498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0121e5-fc0e-4b7e-a4db-260f0718e451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:02.617925 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0121e5-fc0e-4b7e-a4db-260f0718e451 admin admin] PATH_INFO: `/v3/projects/2d698779e6e84255ac335750968491f2` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:02.619315 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1214/2406] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:02.620305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0121e5-fc0e-4b7e-a4db-260f0718e451 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d698779e6e84255ac335750968491f2)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:02.622402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0121e5-fc0e-4b7e-a4db-260f0718e451 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:02.627355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a050aca-6974-4885-89d1-05b0e1b0761c None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:02.647788 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a050aca-6974-4885-89d1-05b0e1b0761c None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:02.678900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8702a0-7b32-4ede-b05d-c8a45f207825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:02.684051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8702a0-7b32-4ede-b05d-c8a45f207825 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:02.687140 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8702a0-7b32-4ede-b05d-c8a45f207825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:02.687401 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8702a0-7b32-4ede-b05d-c8a45f207825 admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:02.688252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8702a0-7b32-4ede-b05d-c8a45f207825 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:02.688435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8702a0-7b32-4ede-b05d-c8a45f207825 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:02.690086 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8702a0-7b32-4ede-b05d-c8a45f207825 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:02.750916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1215/2407] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:02.751549 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8702a0-7b32-4ede-b05d-c8a45f207825 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:02.790569 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8702a0-7b32-4ede-b05d-c8a45f207825 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:02.835147 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-9e0121e5-fc0e-4b7e-a4db-260f0718e451 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d698779e6e84255ac335750968491f2', 'request_id': 'req-9e0121e5-fc0e-4b7e-a4db-260f0718e451'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:02.859229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-9e0121e5-fc0e-4b7e-a4db-260f0718e451 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d698779e6e84255ac335750968491f2', 'request_id': 'req-9e0121e5-fc0e-4b7e-a4db-260f0718e451'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:02.888122 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-9e0121e5-fc0e-4b7e-a4db-260f0718e451 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d698779e6e84255ac335750968491f2', 'request_id': 'req-9e0121e5-fc0e-4b7e-a4db-260f0718e451'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:02.919639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-9e0121e5-fc0e-4b7e-a4db-260f0718e451 admin admin] The token cache is being invalidate because project 2d698779e6e84255ac335750968491f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:15:02.920191 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-9e0121e5-fc0e-4b7e-a4db-260f0718e451 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:02.923073 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-9e0121e5-fc0e-4b7e-a4db-260f0718e451 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:02.926202 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1193/2408] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:15:02 2022] DELETE /identity/v3/projects/2d698779e6e84255ac335750968491f2 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:02.931878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18890baf-7f0a-4ea0-bd99-645b07d6353c None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:02.932476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18890baf-7f0a-4ea0-bd99-645b07d6353c None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:02.932916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18890baf-7f0a-4ea0-bd99-645b07d6353c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:02.989542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f0166-717a-4525-a7ea-8af89ec5e640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:02.989542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f0166-717a-4525-a7ea-8af89ec5e640 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:02.989542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f0166-717a-4525-a7ea-8af89ec5e640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:02.989542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f0166-717a-4525-a7ea-8af89ec5e640 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:02.990747 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f0166-717a-4525-a7ea-8af89ec5e640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:02.990747 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f0166-717a-4525-a7ea-8af89ec5e640 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:02.997257 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-18890baf-7f0a-4ea0-bd99-645b07d6353c None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:02.997257 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-18890baf-7f0a-4ea0-bd99-645b07d6353c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:03.023462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1216/2409] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:15:02 2022] POST /identity/v3/projects => generated 389 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:03.027333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f0166-717a-4525-a7ea-8af89ec5e640 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:03.029821 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-228f0166-717a-4525-a7ea-8af89ec5e640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:03.049092 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1194/2410] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:15:03.053101 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5a350-1a77-44a6-b83f-225b13e57c54 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.053387 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5a350-1a77-44a6-b83f-225b13e57c54 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.053666 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5a350-1a77-44a6-b83f-225b13e57c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.100527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-4cd5a350-1a77-44a6-b83f-225b13e57c54 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:03.101761 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-4cd5a350-1a77-44a6-b83f-225b13e57c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:03.114492 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f0166-717a-4525-a7ea-8af89ec5e640 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:03.120397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb272ab1-6e93-4a50-b503-05c833123518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:03.122577 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb272ab1-6e93-4a50-b503-05c833123518 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.123252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb272ab1-6e93-4a50-b503-05c833123518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.123803 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb272ab1-6e93-4a50-b503-05c833123518 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.124742 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb272ab1-6e93-4a50-b503-05c833123518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:03.126098 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1195/2411] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:15:03.127054 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5a350-1a77-44a6-b83f-225b13e57c54 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:03.128112 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb272ab1-6e93-4a50-b503-05c833123518 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:03.148044 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5a350-1a77-44a6-b83f-225b13e57c54 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:03.160872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44508b-ae89-462e-9490-2773aa8d39e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:03.161382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44508b-ae89-462e-9490-2773aa8d39e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.161905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44508b-ae89-462e-9490-2773aa8d39e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.162654 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44508b-ae89-462e-9490-2773aa8d39e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.164169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44508b-ae89-462e-9490-2773aa8d39e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:03.164553 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44508b-ae89-462e-9490-2773aa8d39e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:03.167187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44508b-ae89-462e-9490-2773aa8d39e9 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:03.226051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1217/2412] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:15:03 2022] POST /identity/v3/users => generated 295 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:03.227266 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb272ab1-6e93-4a50-b503-05c833123518 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:03.228917 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1196/2413] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:03.230090 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44508b-ae89-462e-9490-2773aa8d39e9 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:03.243647 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44508b-ae89-462e-9490-2773aa8d39e9 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:03.253978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb272ab1-6e93-4a50-b503-05c833123518 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:03.260325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecafba-37b4-4de0-955c-3553dd156d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:03.261620 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecafba-37b4-4de0-955c-3553dd156d0b admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.262320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecafba-37b4-4de0-955c-3553dd156d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.262948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecafba-37b4-4de0-955c-3553dd156d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.263896 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecafba-37b4-4de0-955c-3553dd156d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:03.268048 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecafba-37b4-4de0-955c-3553dd156d0b admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:03.270523 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109199b-960d-4d44-ad9d-3f6f93c2ddac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:03.271180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109199b-960d-4d44-ad9d-3f6f93c2ddac admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.273089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109199b-960d-4d44-ad9d-3f6f93c2ddac admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.273386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109199b-960d-4d44-ad9d-3f6f93c2ddac admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.275940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109199b-960d-4d44-ad9d-3f6f93c2ddac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:03.280446 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109199b-960d-4d44-ad9d-3f6f93c2ddac admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:03.341599 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1197/2414] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:15:03.345565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecafba-37b4-4de0-955c-3553dd156d0b admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:03.356866 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecafba-37b4-4de0-955c-3553dd156d0b admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:03.362123 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26ac68-59c4-41a5-92e7-6d98f7a95a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:03.363091 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26ac68-59c4-41a5-92e7-6d98f7a95a46 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.363594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26ac68-59c4-41a5-92e7-6d98f7a95a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.363747 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:03.364094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26ac68-59c4-41a5-92e7-6d98f7a95a46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.364910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26ac68-59c4-41a5-92e7-6d98f7a95a46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:03.367796 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26ac68-59c4-41a5-92e7-6d98f7a95a46 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:03.391217 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1198/2415] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:15:03.393140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26ac68-59c4-41a5-92e7-6d98f7a95a46 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:03.394677 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1218/2416] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:15:03 2022] POST /identity/v3/projects => generated 383 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:03.400198 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109199b-960d-4d44-ad9d-3f6f93c2ddac admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:03.409623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26ac68-59c4-41a5-92e7-6d98f7a95a46 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:03.428365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:03.503443 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109199b-960d-4d44-ad9d-3f6f93c2ddac admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:03.519045 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59657e6e-ab2e-453c-b204-77d13ce7ffb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:03.520010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59657e6e-ab2e-453c-b204-77d13ce7ffb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.520445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59657e6e-ab2e-453c-b204-77d13ce7ffb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.520725 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59657e6e-ab2e-453c-b204-77d13ce7ffb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.521381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59657e6e-ab2e-453c-b204-77d13ce7ffb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:03.523471 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59657e6e-ab2e-453c-b204-77d13ce7ffb6 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:03.524981 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20375832-37c4-454f-8212-f8c7a28ef512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:03.525678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20375832-37c4-454f-8212-f8c7a28ef512 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.525967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20375832-37c4-454f-8212-f8c7a28ef512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.526215 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20375832-37c4-454f-8212-f8c7a28ef512 admin admin] PATH_INFO: `/v3/projects/1f5c3fdccec74d9c82667b96c09c10ef/users/3922dd92178d48b69c237e0e72628d06/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.556265 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1219/2417] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:15:03 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:03.557436 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20375832-37c4-454f-8212-f8c7a28ef512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f5c3fdccec74d9c82667b96c09c10ef, user_id=3922dd92178d48b69c237e0e72628d06, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:03.560643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59657e6e-ab2e-453c-b204-77d13ce7ffb6 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:03.567233 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20375832-37c4-454f-8212-f8c7a28ef512 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:03.584495 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59657e6e-ab2e-453c-b204-77d13ce7ffb6 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:03.593830 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd585f1-74da-43df-8429-922136c7f488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:03.595643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd585f1-74da-43df-8429-922136c7f488 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.597060 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd585f1-74da-43df-8429-922136c7f488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.597568 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd585f1-74da-43df-8429-922136c7f488 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.598999 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd585f1-74da-43df-8429-922136c7f488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:03.602338 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd585f1-74da-43df-8429-922136c7f488 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:03.607495 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1199/2418] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:15:03 2022] PUT /identity/v3/projects/1f5c3fdccec74d9c82667b96c09c10ef/users/3922dd92178d48b69c237e0e72628d06/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:03.618315 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229f1cb-936a-4c76-af50-8fcfafd586b7 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.618565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229f1cb-936a-4c76-af50-8fcfafd586b7 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.618818 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229f1cb-936a-4c76-af50-8fcfafd586b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.627507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1220/2419] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:15:03.631974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd585f1-74da-43df-8429-922136c7f488 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:03.645077 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd585f1-74da-43df-8429-922136c7f488 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:03.707069 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-1229f1cb-936a-4c76-af50-8fcfafd586b7 None None] MFA Rules not processed for user `3922dd92178d48b69c237e0e72628d06`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:03.708644 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-1229f1cb-936a-4c76-af50-8fcfafd586b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:03.742800 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c017a-70b1-4d70-bdfb-5061434931ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:03.744123 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c017a-70b1-4d70-bdfb-5061434931ec admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.744481 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c017a-70b1-4d70-bdfb-5061434931ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.744825 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c017a-70b1-4d70-bdfb-5061434931ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.745727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c017a-70b1-4d70-bdfb-5061434931ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:03.748248 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c017a-70b1-4d70-bdfb-5061434931ec admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:03.762939 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1221/2420] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:15:03.763978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c017a-70b1-4d70-bdfb-5061434931ec admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:03.774596 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c017a-70b1-4d70-bdfb-5061434931ec admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:03.787446 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4597d86-1c81-4ecb-93cd-e916752ba09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:03.788295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4597d86-1c81-4ecb-93cd-e916752ba09e admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.788295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4597d86-1c81-4ecb-93cd-e916752ba09e admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.788636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4597d86-1c81-4ecb-93cd-e916752ba09e admin admin] PATH_INFO: `/v3/projects/f9466597900d4a3ca60a2ef8b7678c87/users/a5e8c33634b246319de495d269c05124/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.811586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4597d86-1c81-4ecb-93cd-e916752ba09e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9466597900d4a3ca60a2ef8b7678c87, user_id=a5e8c33634b246319de495d269c05124, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:03.814627 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4597d86-1c81-4ecb-93cd-e916752ba09e admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:03.841360 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1222/2421] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:15:03 2022] PUT /identity/v3/projects/f9466597900d4a3ca60a2ef8b7678c87/users/a5e8c33634b246319de495d269c05124/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:03.842800 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d7845-c6e4-4355-9310-0de2035a3adf None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.846242 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d7845-c6e4-4355-9310-0de2035a3adf None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.846636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d7845-c6e4-4355-9310-0de2035a3adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.859153 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1200/2422] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:03.860534 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518bd9c-4176-4d2c-b09b-3a6e5bdab3bf None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:03.860823 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518bd9c-4176-4d2c-b09b-3a6e5bdab3bf None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:03.861106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518bd9c-4176-4d2c-b09b-3a6e5bdab3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:03.893170 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-8518bd9c-4176-4d2c-b09b-3a6e5bdab3bf None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:03.894441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8518bd9c-4176-4d2c-b09b-3a6e5bdab3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:03.908754 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-147d7845-c6e4-4355-9310-0de2035a3adf None None] MFA Rules not processed for user `c26232f042844a898c19d1f63152b428`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:03.913876 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-147d7845-c6e4-4355-9310-0de2035a3adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:03.974388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:04.018594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1201/2423] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:04.020783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518bd9c-4176-4d2c-b09b-3a6e5bdab3bf None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.022715 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8518bd9c-4176-4d2c-b09b-3a6e5bdab3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:04.028060 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1223/2424] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:04.028932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d7845-c6e4-4355-9310-0de2035a3adf None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.030307 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-147d7845-c6e4-4355-9310-0de2035a3adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:04.074249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:04.109908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518bd9c-4176-4d2c-b09b-3a6e5bdab3bf None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.116866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d7845-c6e4-4355-9310-0de2035a3adf None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.123132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e1fb7-d9ae-46fd-8cae-953d12b9be25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.123813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e1fb7-d9ae-46fd-8cae-953d12b9be25 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.124092 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e1fb7-d9ae-46fd-8cae-953d12b9be25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.124366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e1fb7-d9ae-46fd-8cae-953d12b9be25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.133197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5639a-b24c-4475-953f-fbc5e2329e4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.133862 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5639a-b24c-4475-953f-fbc5e2329e4a service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.135868 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5639a-b24c-4475-953f-fbc5e2329e4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.136226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5639a-b24c-4475-953f-fbc5e2329e4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.148856 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5639a-b24c-4475-953f-fbc5e2329e4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:04.150754 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5639a-b24c-4475-953f-fbc5e2329e4a service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:04.163923 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1224/2425] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:15:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:15:04.165587 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5639a-b24c-4475-953f-fbc5e2329e4a service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.167000 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-65f5639a-b24c-4475-953f-fbc5e2329e4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:04.185488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5639a-b24c-4475-953f-fbc5e2329e4a service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.190535 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c7aa9-2a1c-440f-9934-077ed9c3ab71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.191173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c7aa9-2a1c-440f-9934-077ed9c3ab71 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.191507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c7aa9-2a1c-440f-9934-077ed9c3ab71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.191786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c7aa9-2a1c-440f-9934-077ed9c3ab71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.192418 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c7aa9-2a1c-440f-9934-077ed9c3ab71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:04.199011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c7aa9-2a1c-440f-9934-077ed9c3ab71 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:04.247359 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e1fb7-d9ae-46fd-8cae-953d12b9be25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:04.249025 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e1fb7-d9ae-46fd-8cae-953d12b9be25 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:04.258065 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1225/2426] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:15:04 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:04.262900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c7aa9-2a1c-440f-9934-077ed9c3ab71 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.275396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1202/2427] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:15:04 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:15:04.395013 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c7aa9-2a1c-440f-9934-077ed9c3ab71 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.407160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713c1b4-5a1a-43d7-b00e-518ffa43e7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.407870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713c1b4-5a1a-43d7-b00e-518ffa43e7e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.408755 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713c1b4-5a1a-43d7-b00e-518ffa43e7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.409036 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713c1b4-5a1a-43d7-b00e-518ffa43e7e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.412323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713c1b4-5a1a-43d7-b00e-518ffa43e7e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:04.416475 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713c1b4-5a1a-43d7-b00e-518ffa43e7e5 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:04.450439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1226/2428] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:15:04 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:04.451931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e1fb7-d9ae-46fd-8cae-953d12b9be25 service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.469737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e1fb7-d9ae-46fd-8cae-953d12b9be25 service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.483979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7001a5-a63a-4e0c-b11b-db6a664c869d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.484636 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7001a5-a63a-4e0c-b11b-db6a664c869d admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.485004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7001a5-a63a-4e0c-b11b-db6a664c869d admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.485351 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7001a5-a63a-4e0c-b11b-db6a664c869d admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.487490 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7001a5-a63a-4e0c-b11b-db6a664c869d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:04.487864 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7001a5-a63a-4e0c-b11b-db6a664c869d admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:04.501473 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1203/2429] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:15:04.506025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713c1b4-5a1a-43d7-b00e-518ffa43e7e5 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.526524 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713c1b4-5a1a-43d7-b00e-518ffa43e7e5 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.535798 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928b458-480c-458d-87cb-c6cbefb80c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.537954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928b458-480c-458d-87cb-c6cbefb80c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.539468 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928b458-480c-458d-87cb-c6cbefb80c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.540493 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928b458-480c-458d-87cb-c6cbefb80c40 admin admin] PATH_INFO: `/v3/projects/f6fa5023538e423290be27a3a2873b69/users/c11e70f0fb9f49dbb4082a5fd35f13fb/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.571292 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928b458-480c-458d-87cb-c6cbefb80c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6fa5023538e423290be27a3a2873b69, user_id=c11e70f0fb9f49dbb4082a5fd35f13fb, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:04.574996 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928b458-480c-458d-87cb-c6cbefb80c40 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:04.605790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1227/2430] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:15:04 2022] PUT /identity/v3/projects/f6fa5023538e423290be27a3a2873b69/users/c11e70f0fb9f49dbb4082a5fd35f13fb/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:15:04.606206 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7001a5-a63a-4e0c-b11b-db6a664c869d admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.617468 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7001a5-a63a-4e0c-b11b-db6a664c869d admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.749441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb97f8c-bb63-4a8e-b160-1a22e0713fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.751999 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb97f8c-bb63-4a8e-b160-1a22e0713fcc admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.752812 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb97f8c-bb63-4a8e-b160-1a22e0713fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.753092 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb97f8c-bb63-4a8e-b160-1a22e0713fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.756035 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb97f8c-bb63-4a8e-b160-1a22e0713fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:04.758395 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb97f8c-bb63-4a8e-b160-1a22e0713fcc admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:04.796415 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1204/2431] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:15:04.797585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928b458-480c-458d-87cb-c6cbefb80c40 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:04.808408 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928b458-480c-458d-87cb-c6cbefb80c40 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:04.825559 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8823d0-ad6a-4db5-97d6-ab8a11837212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:04.826244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8823d0-ad6a-4db5-97d6-ab8a11837212 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:04.826522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8823d0-ad6a-4db5-97d6-ab8a11837212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:04.826793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8823d0-ad6a-4db5-97d6-ab8a11837212 admin admin] PATH_INFO: `/v3/projects/f6fa5023538e423290be27a3a2873b69/users/c11e70f0fb9f49dbb4082a5fd35f13fb/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:04.834094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8823d0-ad6a-4db5-97d6-ab8a11837212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6fa5023538e423290be27a3a2873b69, user_id=c11e70f0fb9f49dbb4082a5fd35f13fb, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:04.836787 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8823d0-ad6a-4db5-97d6-ab8a11837212 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:04.875821 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1228/2432] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:15:04 2022] PUT /identity/v3/projects/f6fa5023538e423290be27a3a2873b69/users/c11e70f0fb9f49dbb4082a5fd35f13fb/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:07.230398 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bce243-7246-4f37-8ee7-0361e184edd6 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:07.231023 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bce243-7246-4f37-8ee7-0361e184edd6 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:07.231023 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bce243-7246-4f37-8ee7-0361e184edd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:07.264658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-b4bce243-7246-4f37-8ee7-0361e184edd6 None None] MFA Rules not processed for user `b3f1ae88e29f4172a7db7393d517a893`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:07.266240 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b4bce243-7246-4f37-8ee7-0361e184edd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:07.407340 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8823d0-ad6a-4db5-97d6-ab8a11837212 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:07.408405 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1205/2433] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 23 07:15:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:15:07.415676 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ee8823d0-ad6a-4db5-97d6-ab8a11837212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:07.436874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:07.589504 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8823d0-ad6a-4db5-97d6-ab8a11837212 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:07.599203 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01c6f4-e0fd-45a2-954b-aed1cbb1ebbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd88ec8b477443af95bff9af03471adf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:07.599962 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01c6f4-e0fd-45a2-954b-aed1cbb1ebbe service swift] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:07.600310 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01c6f4-e0fd-45a2-954b-aed1cbb1ebbe service swift] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:07.600714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01c6f4-e0fd-45a2-954b-aed1cbb1ebbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:07.613479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01c6f4-e0fd-45a2-954b-aed1cbb1ebbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:07.616323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01c6f4-e0fd-45a2-954b-aed1cbb1ebbe service swift] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:07.635136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1229/2434] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 23 07:15:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 07:15:12.878114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bce243-7246-4f37-8ee7-0361e184edd6 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:12.880399 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b4bce243-7246-4f37-8ee7-0361e184edd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:12.920611 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:13.077998 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bce243-7246-4f37-8ee7-0361e184edd6 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:13.101559 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759c576-068d-495b-9492-e8febda68d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:13.102608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759c576-068d-495b-9492-e8febda68d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:13.105135 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759c576-068d-495b-9492-e8febda68d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:13.105819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759c576-068d-495b-9492-e8febda68d50 admin admin] PATH_INFO: `/v3/users/bab4b09414344ef2b2088cf96378f21b` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:13.113587 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759c576-068d-495b-9492-e8febda68d50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bab4b09414344ef2b2088cf96378f21b)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:13.125054 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759c576-068d-495b-9492-e8febda68d50 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:13.342006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-2759c576-068d-495b-9492-e8febda68d50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bab4b09414344ef2b2088cf96378f21b'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:13.342815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-2759c576-068d-495b-9492-e8febda68d50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bab4b09414344ef2b2088cf96378f21b'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:13.343653 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-2759c576-068d-495b-9492-e8febda68d50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bab4b09414344ef2b2088cf96378f21b'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:13.389043 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-2759c576-068d-495b-9492-e8febda68d50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bab4b09414344ef2b2088cf96378f21b'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:13.424059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-2759c576-068d-495b-9492-e8febda68d50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bab4b09414344ef2b2088cf96378f21b'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:13.452678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1206/2435] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:15:12 2022] DELETE /identity/v3/users/bab4b09414344ef2b2088cf96378f21b => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:13.457564 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde65dca-2f74-4e83-913a-696648ac3fab None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:13.458065 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde65dca-2f74-4e83-913a-696648ac3fab None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:13.458174 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde65dca-2f74-4e83-913a-696648ac3fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:13.490495 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-fde65dca-2f74-4e83-913a-696648ac3fab None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:13.491823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-fde65dca-2f74-4e83-913a-696648ac3fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:13.731918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1230/2436] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:13.748015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759c576-068d-495b-9492-e8febda68d50 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:13.749510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2759c576-068d-495b-9492-e8febda68d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:13.811303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f912a-6cb1-41d1-a116-1800fd57752d None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:13.813219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f912a-6cb1-41d1-a116-1800fd57752d None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:13.813677 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f912a-6cb1-41d1-a116-1800fd57752d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:13.877180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-ac3f912a-6cb1-41d1-a116-1800fd57752d None None] MFA Rules not processed for user `a5e8c33634b246319de495d269c05124`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:13.878983 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ac3f912a-6cb1-41d1-a116-1800fd57752d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:13.941467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759c576-068d-495b-9492-e8febda68d50 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:13.948838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fba22-aab7-499a-8739-a0f580e64b02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:13.949682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fba22-aab7-499a-8739-a0f580e64b02 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:13.950099 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fba22-aab7-499a-8739-a0f580e64b02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:13.950398 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fba22-aab7-499a-8739-a0f580e64b02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:13.962560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fba22-aab7-499a-8739-a0f580e64b02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:13.963823 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fba22-aab7-499a-8739-a0f580e64b02 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:13.975472 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1207/2437] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:15:14.057699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1231/2438] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:14.062329 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fba22-aab7-499a-8739-a0f580e64b02 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:14.064233 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-136fba22-aab7-499a-8739-a0f580e64b02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:14.085491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fba22-aab7-499a-8739-a0f580e64b02 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:14.092876 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb3015-6c46-4041-ae57-226810b6b545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:14.093902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb3015-6c46-4041-ae57-226810b6b545 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:14.094522 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb3015-6c46-4041-ae57-226810b6b545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:14.095007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb3015-6c46-4041-ae57-226810b6b545 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:14.095875 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb3015-6c46-4041-ae57-226810b6b545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:14.098108 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb3015-6c46-4041-ae57-226810b6b545 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:14.157164 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1208/2439] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:15:14 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:14.163325 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f912a-6cb1-41d1-a116-1800fd57752d None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:14.181795 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f912a-6cb1-41d1-a116-1800fd57752d None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:14.262067 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:14.336237 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ea34f-5a3a-4b85-85be-0b66f114af6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:14.338563 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ea34f-5a3a-4b85-85be-0b66f114af6b admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:14.339666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ea34f-5a3a-4b85-85be-0b66f114af6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:14.340705 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ea34f-5a3a-4b85-85be-0b66f114af6b admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:14.342337 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ea34f-5a3a-4b85-85be-0b66f114af6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:14.344657 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ea34f-5a3a-4b85-85be-0b66f114af6b admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:14.379769 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1232/2440] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:15:14 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:14.383360 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb3015-6c46-4041-ae57-226810b6b545 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:14.395783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb3015-6c46-4041-ae57-226810b6b545 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:14.411320 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46414e6-f4a0-4213-85ac-0e2317b51fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:14.412387 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46414e6-f4a0-4213-85ac-0e2317b51fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:14.412657 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46414e6-f4a0-4213-85ac-0e2317b51fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:14.412832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46414e6-f4a0-4213-85ac-0e2317b51fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:14.413512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46414e6-f4a0-4213-85ac-0e2317b51fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:14.415150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46414e6-f4a0-4213-85ac-0e2317b51fc7 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:14.427479 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1209/2441] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:15:14.433077 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ea34f-5a3a-4b85-85be-0b66f114af6b admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:14.445865 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ea34f-5a3a-4b85-85be-0b66f114af6b admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:14.452457 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e9f29-1880-4f5a-b9c3-e71dca0147c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:14.453435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e9f29-1880-4f5a-b9c3-e71dca0147c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:14.453908 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e9f29-1880-4f5a-b9c3-e71dca0147c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:14.454414 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e9f29-1880-4f5a-b9c3-e71dca0147c3 admin admin] PATH_INFO: `/v3/projects/e6b6e9084ed6444aa677efceac802ec0/users/3751f05926f7439984f0c564d20b4086/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:14.477479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e9f29-1880-4f5a-b9c3-e71dca0147c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6b6e9084ed6444aa677efceac802ec0, user_id=3751f05926f7439984f0c564d20b4086, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:14.485695 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e9f29-1880-4f5a-b9c3-e71dca0147c3 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:14.510228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1233/2442] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:15:14 2022] PUT /identity/v3/projects/e6b6e9084ed6444aa677efceac802ec0/users/3751f05926f7439984f0c564d20b4086/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:14.514300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46414e6-f4a0-4213-85ac-0e2317b51fc7 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:14.523247 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:14.533039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46414e6-f4a0-4213-85ac-0e2317b51fc7 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:14.666794 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bb78d-0554-4fc3-a114-fd97ab309fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:14.667463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bb78d-0554-4fc3-a114-fd97ab309fbe admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:14.667742 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bb78d-0554-4fc3-a114-fd97ab309fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:14.668073 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bb78d-0554-4fc3-a114-fd97ab309fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:14.668745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bb78d-0554-4fc3-a114-fd97ab309fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:14.672513 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bb78d-0554-4fc3-a114-fd97ab309fbe admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:14.705478 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1210/2443] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:15:14.710094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e9f29-1880-4f5a-b9c3-e71dca0147c3 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:14.737032 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e9f29-1880-4f5a-b9c3-e71dca0147c3 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:14.749367 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecd867-81e5-4110-b011-7a42f9a37492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:14.750065 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecd867-81e5-4110-b011-7a42f9a37492 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:14.750644 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecd867-81e5-4110-b011-7a42f9a37492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:14.750644 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecd867-81e5-4110-b011-7a42f9a37492 admin admin] PATH_INFO: `/v3/projects/e6b6e9084ed6444aa677efceac802ec0/users/3751f05926f7439984f0c564d20b4086/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:14.772459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecd867-81e5-4110-b011-7a42f9a37492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6b6e9084ed6444aa677efceac802ec0, user_id=3751f05926f7439984f0c564d20b4086, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:14.775204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecd867-81e5-4110-b011-7a42f9a37492 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:14.792255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1234/2444] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:15:14 2022] PUT /identity/v3/projects/e6b6e9084ed6444aa677efceac802ec0/users/3751f05926f7439984f0c564d20b4086/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:15.174080 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab29cc0-2670-4c43-8b84-45935e4c5cad None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:15.174912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab29cc0-2670-4c43-8b84-45935e4c5cad None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:15.175201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab29cc0-2670-4c43-8b84-45935e4c5cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:15.250147 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-aab29cc0-2670-4c43-8b84-45935e4c5cad None None] MFA Rules not processed for user `c11e70f0fb9f49dbb4082a5fd35f13fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:15.251870 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-aab29cc0-2670-4c43-8b84-45935e4c5cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:15.373218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecd867-81e5-4110-b011-7a42f9a37492 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:15.375171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-70ecd867-81e5-4110-b011-7a42f9a37492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:15.419546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1211/2445] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:15.429006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab29cc0-2670-4c43-8b84-45935e4c5cad None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:15.430795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-aab29cc0-2670-4c43-8b84-45935e4c5cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:15.504628 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecd867-81e5-4110-b011-7a42f9a37492 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:15.525760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992049c3-7439-4ef8-9d96-fd54e79de668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:15.527240 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992049c3-7439-4ef8-9d96-fd54e79de668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:15.528859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992049c3-7439-4ef8-9d96-fd54e79de668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:15.529479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992049c3-7439-4ef8-9d96-fd54e79de668 admin admin] PATH_INFO: `/v3/projects/6e209b5695624d0e84f93d375c7fe1ab` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:15.531144 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992049c3-7439-4ef8-9d96-fd54e79de668 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e209b5695624d0e84f93d375c7fe1ab)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:15.535967 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992049c3-7439-4ef8-9d96-fd54e79de668 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:15.645033 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab29cc0-2670-4c43-8b84-45935e4c5cad None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:15.662092 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdb043-4c91-475d-83f2-220960daa066 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:15.662447 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdb043-4c91-475d-83f2-220960daa066 service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:15.663130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdb043-4c91-475d-83f2-220960daa066 service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:15.663130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdb043-4c91-475d-83f2-220960daa066 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:15.673011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-adfdb043-4c91-475d-83f2-220960daa066 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:15.710667 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-992049c3-7439-4ef8-9d96-fd54e79de668 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e209b5695624d0e84f93d375c7fe1ab', 'request_id': 'req-992049c3-7439-4ef8-9d96-fd54e79de668'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:15.725322 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-992049c3-7439-4ef8-9d96-fd54e79de668 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e209b5695624d0e84f93d375c7fe1ab', 'request_id': 'req-992049c3-7439-4ef8-9d96-fd54e79de668'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:15.732754 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:15.740447 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-992049c3-7439-4ef8-9d96-fd54e79de668 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e209b5695624d0e84f93d375c7fe1ab', 'request_id': 'req-992049c3-7439-4ef8-9d96-fd54e79de668'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:15.778269 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-992049c3-7439-4ef8-9d96-fd54e79de668 admin admin] The token cache is being invalidate because project 6e209b5695624d0e84f93d375c7fe1ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:15:15.779788 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-992049c3-7439-4ef8-9d96-fd54e79de668 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:15.783314 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-992049c3-7439-4ef8-9d96-fd54e79de668 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:15.795029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1235/2446] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:15:15 2022] DELETE /identity/v3/projects/6e209b5695624d0e84f93d375c7fe1ab => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:15.801994 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e456ff-0623-4130-b4e8-074c46b603ce None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:15.802291 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e456ff-0623-4130-b4e8-074c46b603ce None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:15.802572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e456ff-0623-4130-b4e8-074c46b603ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:15.813638 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfdb043-4c91-475d-83f2-220960daa066 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:15.823759 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfdb043-4c91-475d-83f2-220960daa066 service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:15.840154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1212/2447] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:15:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:15:15.855791 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-86e456ff-0623-4130-b4e8-074c46b603ce None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:15.857503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-86e456ff-0623-4130-b4e8-074c46b603ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:15.865855 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:15.992698 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1236/2448] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:16.001446 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b25c7f-33a2-4161-9b67-8fc297b5ad76 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.001820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b25c7f-33a2-4161-9b67-8fc297b5ad76 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.002020 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b25c7f-33a2-4161-9b67-8fc297b5ad76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.061374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-07b25c7f-33a2-4161-9b67-8fc297b5ad76 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:16.062900 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-07b25c7f-33a2-4161-9b67-8fc297b5ad76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:16.086156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1213/2449] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:16.090808 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e456ff-0623-4130-b4e8-074c46b603ce None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:16.108283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e456ff-0623-4130-b4e8-074c46b603ce None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:16.120160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4320e3-d489-48d2-94b6-374f0c64c892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:16.121119 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4320e3-d489-48d2-94b6-374f0c64c892 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.121579 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4320e3-d489-48d2-94b6-374f0c64c892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.122001 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4320e3-d489-48d2-94b6-374f0c64c892 admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.122811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4320e3-d489-48d2-94b6-374f0c64c892 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:15:16.123258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4320e3-d489-48d2-94b6-374f0c64c892 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:16.125099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4320e3-d489-48d2-94b6-374f0c64c892 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:16.215833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1237/2450] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:15:16.216308 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b25c7f-33a2-4161-9b67-8fc297b5ad76 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:16.240171 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b25c7f-33a2-4161-9b67-8fc297b5ad76 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:16.282012 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43460fe1-ebc8-4cc0-bc65-2eb87de33cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:16.282663 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43460fe1-ebc8-4cc0-bc65-2eb87de33cee admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.282941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43460fe1-ebc8-4cc0-bc65-2eb87de33cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.283210 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43460fe1-ebc8-4cc0-bc65-2eb87de33cee admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.284077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43460fe1-ebc8-4cc0-bc65-2eb87de33cee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:16.286263 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43460fe1-ebc8-4cc0-bc65-2eb87de33cee admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:16.346447 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1214/2451] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:15:16 2022] POST /identity/v3/projects => generated 379 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:16.350782 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4320e3-d489-48d2-94b6-374f0c64c892 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:16.362774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4320e3-d489-48d2-94b6-374f0c64c892 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:16.435430 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f382a0-96e3-4599-8fde-cdafa683026d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:16.436178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f382a0-96e3-4599-8fde-cdafa683026d admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.436737 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f382a0-96e3-4599-8fde-cdafa683026d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.437111 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f382a0-96e3-4599-8fde-cdafa683026d admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.437856 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f382a0-96e3-4599-8fde-cdafa683026d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:16.441332 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f382a0-96e3-4599-8fde-cdafa683026d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:16.486289 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43460fe1-ebc8-4cc0-bc65-2eb87de33cee admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:16.498153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1238/2452] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:15:16 2022] POST /identity/v3/users => generated 290 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:16.516604 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43460fe1-ebc8-4cc0-bc65-2eb87de33cee admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:16.532930 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe21e5-2209-4031-8db6-6f8bc386ccce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:16.533799 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe21e5-2209-4031-8db6-6f8bc386ccce admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.534184 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe21e5-2209-4031-8db6-6f8bc386ccce admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.534610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe21e5-2209-4031-8db6-6f8bc386ccce admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.535438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe21e5-2209-4031-8db6-6f8bc386ccce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:16.537754 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe21e5-2209-4031-8db6-6f8bc386ccce admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:16.577681 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1215/2453] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:15:16.582296 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f382a0-96e3-4599-8fde-cdafa683026d admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:16.600718 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f382a0-96e3-4599-8fde-cdafa683026d admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:16.640293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63676195-b5d3-44f2-9d14-1ac2efb6cf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:16.649479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63676195-b5d3-44f2-9d14-1ac2efb6cf3c admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.649767 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63676195-b5d3-44f2-9d14-1ac2efb6cf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.650297 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63676195-b5d3-44f2-9d14-1ac2efb6cf3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.655188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63676195-b5d3-44f2-9d14-1ac2efb6cf3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:16.658816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63676195-b5d3-44f2-9d14-1ac2efb6cf3c admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:16.713716 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1239/2454] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:15:16.718417 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe21e5-2209-4031-8db6-6f8bc386ccce admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:16.735318 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe21e5-2209-4031-8db6-6f8bc386ccce admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:16.748187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4bab4-34f9-4381-9ac7-27c8cd208b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:16.754378 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4bab4-34f9-4381-9ac7-27c8cd208b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.756735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4bab4-34f9-4381-9ac7-27c8cd208b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.759194 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4bab4-34f9-4381-9ac7-27c8cd208b6b admin admin] PATH_INFO: `/v3/projects/f43b271d00f74eb7aa3d0719315c5a99/users/ffd12f09606b4e6ba12f63b976df1884/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.828526 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4bab4-34f9-4381-9ac7-27c8cd208b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f43b271d00f74eb7aa3d0719315c5a99, user_id=ffd12f09606b4e6ba12f63b976df1884, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:16.835203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4bab4-34f9-4381-9ac7-27c8cd208b6b admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:16.865520 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1216/2455] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:15:16 2022] PUT /identity/v3/projects/f43b271d00f74eb7aa3d0719315c5a99/users/ffd12f09606b4e6ba12f63b976df1884/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:16.871788 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def34463-6d02-46b0-a657-260094aa97b2 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:16.872104 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def34463-6d02-46b0-a657-260094aa97b2 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:16.872429 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def34463-6d02-46b0-a657-260094aa97b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:16.975449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-def34463-6d02-46b0-a657-260094aa97b2 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:16.985089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-def34463-6d02-46b0-a657-260094aa97b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:17.129496 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1240/2456] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:17.183076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4bab4-34f9-4381-9ac7-27c8cd208b6b admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:17.185448 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-28e4bab4-34f9-4381-9ac7-27c8cd208b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:17.319643 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4bab4-34f9-4381-9ac7-27c8cd208b6b admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:17.325129 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97d367-09aa-4588-97e0-d467a6cc5290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:17.326113 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97d367-09aa-4588-97e0-d467a6cc5290 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:17.326739 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97d367-09aa-4588-97e0-d467a6cc5290 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:17.327222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97d367-09aa-4588-97e0-d467a6cc5290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:17.345623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97d367-09aa-4588-97e0-d467a6cc5290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:17.348658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97d367-09aa-4588-97e0-d467a6cc5290 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:17.390517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1217/2457] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:15:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:15:18.351925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014b13f-e2a4-4803-bb63-fe9255a1bc3f None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:18.352473 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014b13f-e2a4-4803-bb63-fe9255a1bc3f None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:18.352473 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014b13f-e2a4-4803-bb63-fe9255a1bc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:18.412823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-7014b13f-e2a4-4803-bb63-fe9255a1bc3f None None] MFA Rules not processed for user `c26232f042844a898c19d1f63152b428`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:18.414883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-7014b13f-e2a4-4803-bb63-fe9255a1bc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:18.450967 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:18.565569 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1241/2458] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:27.011095 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace96a27-f2ef-4fbe-b7c4-d1c7a77db935 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:27.011776 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace96a27-f2ef-4fbe-b7c4-d1c7a77db935 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:27.011776 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace96a27-f2ef-4fbe-b7c4-d1c7a77db935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:27.116724 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-ace96a27-f2ef-4fbe-b7c4-d1c7a77db935 None None] MFA Rules not processed for user `ffd12f09606b4e6ba12f63b976df1884`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:27.118778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ace96a27-f2ef-4fbe-b7c4-d1c7a77db935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:27.234849 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:27.291227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1218/2459] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:27.296339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014b13f-e2a4-4803-bb63-fe9255a1bc3f None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:27.334005 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014b13f-e2a4-4803-bb63-fe9255a1bc3f None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:27.352950 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c011531-23d2-4c90-902e-ccc5237fb2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:27.353964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c011531-23d2-4c90-902e-ccc5237fb2f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:27.357054 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c011531-23d2-4c90-902e-ccc5237fb2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:27.357769 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c011531-23d2-4c90-902e-ccc5237fb2f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:27.361864 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c011531-23d2-4c90-902e-ccc5237fb2f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:27.369889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace96a27-f2ef-4fbe-b7c4-d1c7a77db935 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:27.372083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ace96a27-f2ef-4fbe-b7c4-d1c7a77db935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:27.373005 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c011531-23d2-4c90-902e-ccc5237fb2f8 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:27.419795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace96a27-f2ef-4fbe-b7c4-d1c7a77db935 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:27.441300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706432b-4626-44e1-aaed-eb89ac4fca86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:27.442077 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706432b-4626-44e1-aaed-eb89ac4fca86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:27.442439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706432b-4626-44e1-aaed-eb89ac4fca86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:27.442786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706432b-4626-44e1-aaed-eb89ac4fca86 admin admin] PATH_INFO: `/v3/users/3922dd92178d48b69c237e0e72628d06` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:27.453786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4706432b-4626-44e1-aaed-eb89ac4fca86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3922dd92178d48b69c237e0e72628d06)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:27.457201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4706432b-4626-44e1-aaed-eb89ac4fca86 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:27.561466 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1242/2460] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:15:27 2022] POST /identity/v3/projects => generated 377 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:27.566124 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c011531-23d2-4c90-902e-ccc5237fb2f8 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:27.616723 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4706432b-4626-44e1-aaed-eb89ac4fca86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3922dd92178d48b69c237e0e72628d06'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:27.617477 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4706432b-4626-44e1-aaed-eb89ac4fca86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3922dd92178d48b69c237e0e72628d06'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:27.618200 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4706432b-4626-44e1-aaed-eb89ac4fca86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3922dd92178d48b69c237e0e72628d06'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:27.666128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4706432b-4626-44e1-aaed-eb89ac4fca86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3922dd92178d48b69c237e0e72628d06'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:27.701667 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4706432b-4626-44e1-aaed-eb89ac4fca86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3922dd92178d48b69c237e0e72628d06'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:27.725225 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1219/2461] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:15:27 2022] DELETE /identity/v3/users/3922dd92178d48b69c237e0e72628d06 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:15:27.731061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e463a9e-3460-4974-b538-50cb993714b0 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:27.731394 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e463a9e-3460-4974-b538-50cb993714b0 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:27.731782 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e463a9e-3460-4974-b538-50cb993714b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:27.803133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-0e463a9e-3460-4974-b538-50cb993714b0 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:27.822939 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-0e463a9e-3460-4974-b538-50cb993714b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:27.890470 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c011531-23d2-4c90-902e-ccc5237fb2f8 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:27.990794 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1220/2462] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:28.003912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e463a9e-3460-4974-b538-50cb993714b0 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:28.005426 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-0e463a9e-3460-4974-b538-50cb993714b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:28.063001 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2bafd8-8a09-4d87-b7e9-e459c68b31d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:28.063731 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bafd8-8a09-4d87-b7e9-e459c68b31d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.063981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bafd8-8a09-4d87-b7e9-e459c68b31d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.064252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bafd8-8a09-4d87-b7e9-e459c68b31d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.066818 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2bafd8-8a09-4d87-b7e9-e459c68b31d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:28.068860 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2bafd8-8a09-4d87-b7e9-e459c68b31d9 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:28.131855 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1243/2463] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:15:27 2022] POST /identity/v3/users => generated 295 bytes in 566 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:15:28.136548 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2bafd8-8a09-4d87-b7e9-e459c68b31d9 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:28.145139 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-fc2bafd8-8a09-4d87-b7e9-e459c68b31d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:28.173013 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2bafd8-8a09-4d87-b7e9-e459c68b31d9 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:28.175137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e463a9e-3460-4974-b538-50cb993714b0 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:28.187842 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b2cac-ff24-48ca-83a3-6ac057588366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:28.188516 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b2cac-ff24-48ca-83a3-6ac057588366 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.188813 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b2cac-ff24-48ca-83a3-6ac057588366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.189069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b2cac-ff24-48ca-83a3-6ac057588366 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.189728 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b2cac-ff24-48ca-83a3-6ac057588366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:28.191462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b2cac-ff24-48ca-83a3-6ac057588366 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:28.192335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efca8d-2847-434d-98b9-7044575fee76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:28.193208 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efca8d-2847-434d-98b9-7044575fee76 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.194150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efca8d-2847-434d-98b9-7044575fee76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.194586 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efca8d-2847-434d-98b9-7044575fee76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.218993 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efca8d-2847-434d-98b9-7044575fee76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:28.221101 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efca8d-2847-434d-98b9-7044575fee76 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:28.255325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1221/2464] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:15:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:15:28.263562 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efca8d-2847-434d-98b9-7044575fee76 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:28.267870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1244/2465] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:15:28.285900 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efca8d-2847-434d-98b9-7044575fee76 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:28.297969 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eda559-94ea-43fc-8310-1a4d573034a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:28.301875 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eda559-94ea-43fc-8310-1a4d573034a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.302322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eda559-94ea-43fc-8310-1a4d573034a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.303013 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eda559-94ea-43fc-8310-1a4d573034a3 admin admin] PATH_INFO: `/v3/projects/5d08e1b2b59d46499716a4c4d55e2b0c/users/6a9b12585539482c945df41a8b6b6420/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.307738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:28.352777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eda559-94ea-43fc-8310-1a4d573034a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d08e1b2b59d46499716a4c4d55e2b0c, user_id=6a9b12585539482c945df41a8b6b6420, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:28.356420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eda559-94ea-43fc-8310-1a4d573034a3 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:28.406815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1222/2466] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:15:28 2022] PUT /identity/v3/projects/5d08e1b2b59d46499716a4c4d55e2b0c/users/6a9b12585539482c945df41a8b6b6420/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:28.410401 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b2cac-ff24-48ca-83a3-6ac057588366 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:28.433330 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b2cac-ff24-48ca-83a3-6ac057588366 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:28.553179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:28.691336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3cbcc-6441-48e1-895c-4819210fd192 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.692144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3cbcc-6441-48e1-895c-4819210fd192 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.692845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3cbcc-6441-48e1-895c-4819210fd192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.696231 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e910e-ccab-41ce-85d8-1990662fba6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:28.696895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e910e-ccab-41ce-85d8-1990662fba6a admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.697288 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e910e-ccab-41ce-85d8-1990662fba6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.697612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e910e-ccab-41ce-85d8-1990662fba6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.698306 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e910e-ccab-41ce-85d8-1990662fba6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:28.703844 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e910e-ccab-41ce-85d8-1990662fba6a admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:28.733325 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1245/2467] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:15:28.735851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e910e-ccab-41ce-85d8-1990662fba6a admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:28.780854 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e910e-ccab-41ce-85d8-1990662fba6a admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:28.795009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f3200-9961-4ff3-8ff0-40c13ac819a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:28.796051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f3200-9961-4ff3-8ff0-40c13ac819a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:28.796501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f3200-9961-4ff3-8ff0-40c13ac819a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:28.796921 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f3200-9961-4ff3-8ff0-40c13ac819a2 admin admin] PATH_INFO: `/v3/projects/5d08e1b2b59d46499716a4c4d55e2b0c/users/6a9b12585539482c945df41a8b6b6420/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:28.801801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f3200-9961-4ff3-8ff0-40c13ac819a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d08e1b2b59d46499716a4c4d55e2b0c, user_id=6a9b12585539482c945df41a8b6b6420, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:28.814096 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-e6e3cbcc-6441-48e1-895c-4819210fd192 None None] MFA Rules not processed for user `3751f05926f7439984f0c564d20b4086`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:28.815678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e6e3cbcc-6441-48e1-895c-4819210fd192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:28.817512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f3200-9961-4ff3-8ff0-40c13ac819a2 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:28.854721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1246/2468] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:15:28 2022] PUT /identity/v3/projects/5d08e1b2b59d46499716a4c4d55e2b0c/users/6a9b12585539482c945df41a8b6b6420/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:28.993933 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1223/2469] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:29.138433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f3200-9961-4ff3-8ff0-40c13ac819a2 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:29.143890 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ef0f3200-9961-4ff3-8ff0-40c13ac819a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:29.373322 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f3200-9961-4ff3-8ff0-40c13ac819a2 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:29.382092 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5df56-e0f1-4d45-bd41-9a0ecb169eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:29.382092 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5df56-e0f1-4d45-bd41-9a0ecb169eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:29.382092 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5df56-e0f1-4d45-bd41-9a0ecb169eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:29.382092 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5df56-e0f1-4d45-bd41-9a0ecb169eda admin admin] PATH_INFO: `/v3/projects/1f5c3fdccec74d9c82667b96c09c10ef` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:29.399508 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f5df56-e0f1-4d45-bd41-9a0ecb169eda admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f5c3fdccec74d9c82667b96c09c10ef)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:29.401654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f5df56-e0f1-4d45-bd41-9a0ecb169eda admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:29.476919 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f0f5df56-e0f1-4d45-bd41-9a0ecb169eda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f5c3fdccec74d9c82667b96c09c10ef', 'request_id': 'req-f0f5df56-e0f1-4d45-bd41-9a0ecb169eda'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:29.500183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f0f5df56-e0f1-4d45-bd41-9a0ecb169eda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f5c3fdccec74d9c82667b96c09c10ef', 'request_id': 'req-f0f5df56-e0f1-4d45-bd41-9a0ecb169eda'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:29.526724 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f0f5df56-e0f1-4d45-bd41-9a0ecb169eda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f5c3fdccec74d9c82667b96c09c10ef', 'request_id': 'req-f0f5df56-e0f1-4d45-bd41-9a0ecb169eda'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:29.554363 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f0f5df56-e0f1-4d45-bd41-9a0ecb169eda admin admin] The token cache is being invalidate because project 1f5c3fdccec74d9c82667b96c09c10ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:15:29.554699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f0f5df56-e0f1-4d45-bd41-9a0ecb169eda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:29.555682 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f0f5df56-e0f1-4d45-bd41-9a0ecb169eda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:29.578714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1247/2470] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:15:29 2022] DELETE /identity/v3/projects/1f5c3fdccec74d9c82667b96c09c10ef => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:33.333489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3cbcc-6441-48e1-895c-4819210fd192 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:33.335898 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e6e3cbcc-6441-48e1-895c-4819210fd192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:33.549560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3cbcc-6441-48e1-895c-4819210fd192 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:33.683093 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fed2cf-42d7-40db-8103-d7e270d28c9e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57a5f04cb9674b86a6b4b9a472e1afc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:33.684106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fed2cf-42d7-40db-8103-d7e270d28c9e service glance] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:33.684411 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fed2cf-42d7-40db-8103-d7e270d28c9e service glance] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:33.684763 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fed2cf-42d7-40db-8103-d7e270d28c9e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:33.686880 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e0fed2cf-42d7-40db-8103-d7e270d28c9e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:33.742528 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:33.772511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fed2cf-42d7-40db-8103-d7e270d28c9e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:33.774073 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fed2cf-42d7-40db-8103-d7e270d28c9e service glance] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:33.790881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1224/2471] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:15:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:15:34.107109 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643adfe-2c86-470f-8764-75f035bdee9e None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:34.108157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643adfe-2c86-470f-8764-75f035bdee9e None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:34.108591 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643adfe-2c86-470f-8764-75f035bdee9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:34.173071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-e643adfe-2c86-470f-8764-75f035bdee9e None None] MFA Rules not processed for user `b3f1ae88e29f4172a7db7393d517a893`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:34.187835 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e643adfe-2c86-470f-8764-75f035bdee9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:34.344531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:34.402260 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1248/2472] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 23 07:15:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:34.425326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fed2cf-42d7-40db-8103-d7e270d28c9e service glance] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:34.426913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e0fed2cf-42d7-40db-8103-d7e270d28c9e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:34.595884 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fed2cf-42d7-40db-8103-d7e270d28c9e service glance] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:34.622167 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33b12a-ae53-47ef-8821-2bed8ba68a4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd88ec8b477443af95bff9af03471adf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:34.623163 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33b12a-ae53-47ef-8821-2bed8ba68a4a service swift] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:34.623644 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33b12a-ae53-47ef-8821-2bed8ba68a4a service swift] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:34.624092 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33b12a-ae53-47ef-8821-2bed8ba68a4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:34.643275 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33b12a-ae53-47ef-8821-2bed8ba68a4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:34.647365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33b12a-ae53-47ef-8821-2bed8ba68a4a service swift] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:34.700394 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1225/2473] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 23 07:15:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 07:15:36.591883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f0c8a-66e7-40f5-8c2a-b06a4c1ad127 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:36.592953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f0c8a-66e7-40f5-8c2a-b06a4c1ad127 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:36.593246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f0c8a-66e7-40f5-8c2a-b06a4c1ad127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:36.680842 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-ad7f0c8a-66e7-40f5-8c2a-b06a4c1ad127 None None] MFA Rules not processed for user `6a9b12585539482c945df41a8b6b6420`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:36.680842 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ad7f0c8a-66e7-40f5-8c2a-b06a4c1ad127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:37.039691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1249/2474] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:37.062749 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33b12a-ae53-47ef-8821-2bed8ba68a4a service swift] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:37.064610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-1d33b12a-ae53-47ef-8821-2bed8ba68a4a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:37.267590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33b12a-ae53-47ef-8821-2bed8ba68a4a service swift] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:37.283966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f37f1-9fa0-47a2-8a9b-d97ff9bbf373 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:37.284676 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f37f1-9fa0-47a2-8a9b-d97ff9bbf373 service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:37.284956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f37f1-9fa0-47a2-8a9b-d97ff9bbf373 service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:37.285231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f37f1-9fa0-47a2-8a9b-d97ff9bbf373 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:37.306487 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-127f37f1-9fa0-47a2-8a9b-d97ff9bbf373 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:37.460421 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127f37f1-9fa0-47a2-8a9b-d97ff9bbf373 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:37.462532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127f37f1-9fa0-47a2-8a9b-d97ff9bbf373 service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:37.499767 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1226/2475] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:15:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:15:45.164712 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f0c8a-66e7-40f5-8c2a-b06a4c1ad127 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:45.170019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ad7f0c8a-66e7-40f5-8c2a-b06a4c1ad127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:45.361366 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f0c8a-66e7-40f5-8c2a-b06a4c1ad127 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:45.370067 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acb31c-e00c-4df2-a1ca-d8719aef2136 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9b0f54c988e43bda474f978a69fa4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:45.371237 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acb31c-e00c-4df2-a1ca-d8719aef2136 service placement] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:45.371829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acb31c-e00c-4df2-a1ca-d8719aef2136 service placement] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:45.372358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acb31c-e00c-4df2-a1ca-d8719aef2136 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:45.374727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b2acb31c-e00c-4df2-a1ca-d8719aef2136 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:45.389978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acb31c-e00c-4df2-a1ca-d8719aef2136 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:45.392117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acb31c-e00c-4df2-a1ca-d8719aef2136 service placement] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:45.427474 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1250/2476] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:15:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:15:47.052670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa0ff9-8efa-4aad-b383-c299c162fdce None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:47.053205 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa0ff9-8efa-4aad-b383-c299c162fdce None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:47.053205 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa0ff9-8efa-4aad-b383-c299c162fdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:47.067528 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:47.110603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-e2aa0ff9-8efa-4aad-b383-c299c162fdce None None] MFA Rules not processed for user `b3f1ae88e29f4172a7db7393d517a893`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:47.115076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e2aa0ff9-8efa-4aad-b383-c299c162fdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:47.173216 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1227/2477] 10.222.0.17 () {58 vars in 967 bytes} [Tue Aug 23 07:15:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:47.282824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acb31c-e00c-4df2-a1ca-d8719aef2136 service placement] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:47.303950 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acb31c-e00c-4df2-a1ca-d8719aef2136 service placement] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:47.321905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57254ef3-11cb-4e50-8c1d-15241845ac0b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd88ec8b477443af95bff9af03471adf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:47.322867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57254ef3-11cb-4e50-8c1d-15241845ac0b service swift] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:47.323312 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57254ef3-11cb-4e50-8c1d-15241845ac0b service swift] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:47.323768 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57254ef3-11cb-4e50-8c1d-15241845ac0b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:47.339694 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57254ef3-11cb-4e50-8c1d-15241845ac0b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:47.342347 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57254ef3-11cb-4e50-8c1d-15241845ac0b service swift] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:47.361502 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1251/2478] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 23 07:15:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 07:15:50.314262 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa0ff9-8efa-4aad-b383-c299c162fdce None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:50.316014 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e2aa0ff9-8efa-4aad-b383-c299c162fdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:50.436071 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa0ff9-8efa-4aad-b383-c299c162fdce None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:50.445445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347edd3-3ced-4a6b-8868-dc0a3b2f5450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:50.446571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347edd3-3ced-4a6b-8868-dc0a3b2f5450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:50.447006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347edd3-3ced-4a6b-8868-dc0a3b2f5450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:50.447578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347edd3-3ced-4a6b-8868-dc0a3b2f5450 admin admin] PATH_INFO: `/v3/users/9f4d65ffaab942f29d449b64805c765a` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:50.449071 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347edd3-3ced-4a6b-8868-dc0a3b2f5450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f4d65ffaab942f29d449b64805c765a)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:50.451662 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347edd3-3ced-4a6b-8868-dc0a3b2f5450 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:50.554490 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4347edd3-3ced-4a6b-8868-dc0a3b2f5450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f4d65ffaab942f29d449b64805c765a'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:50.555427 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4347edd3-3ced-4a6b-8868-dc0a3b2f5450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f4d65ffaab942f29d449b64805c765a'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:50.556165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4347edd3-3ced-4a6b-8868-dc0a3b2f5450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f4d65ffaab942f29d449b64805c765a'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:50.565435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4347edd3-3ced-4a6b-8868-dc0a3b2f5450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f4d65ffaab942f29d449b64805c765a'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:50.589507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4347edd3-3ced-4a6b-8868-dc0a3b2f5450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f4d65ffaab942f29d449b64805c765a'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:50.617143 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1228/2479] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:15:50 2022] DELETE /identity/v3/users/9f4d65ffaab942f29d449b64805c765a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:15:50.864968 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9021d7d-4ef9-41ab-b89e-3d2c7b7b95e0 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:50.865944 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9021d7d-4ef9-41ab-b89e-3d2c7b7b95e0 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:50.866218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9021d7d-4ef9-41ab-b89e-3d2c7b7b95e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:50.877464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347edd3-3ced-4a6b-8868-dc0a3b2f5450 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:50.879294 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-4347edd3-3ced-4a6b-8868-dc0a3b2f5450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:50.887264 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-c9021d7d-4ef9-41ab-b89e-3d2c7b7b95e0 None None] MFA Rules not processed for user `a5e8c33634b246319de495d269c05124`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:50.889220 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c9021d7d-4ef9-41ab-b89e-3d2c7b7b95e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:50.946192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:50.970334 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1252/2480] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:50.999243 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347edd3-3ced-4a6b-8868-dc0a3b2f5450 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:51.004188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160d37c-93e3-46f8-b9e5-e824b4a369e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:51.004899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160d37c-93e3-46f8-b9e5-e824b4a369e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:51.005421 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160d37c-93e3-46f8-b9e5-e824b4a369e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:51.005684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160d37c-93e3-46f8-b9e5-e824b4a369e0 admin admin] PATH_INFO: `/v3/projects/0a2ef3b6384a4194a26ba51078c425af` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:51.006874 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160d37c-93e3-46f8-b9e5-e824b4a369e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a2ef3b6384a4194a26ba51078c425af)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:51.008871 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160d37c-93e3-46f8-b9e5-e824b4a369e0 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:51.033921 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:15:51.085139 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-7160d37c-93e3-46f8-b9e5-e824b4a369e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a2ef3b6384a4194a26ba51078c425af', 'request_id': 'req-7160d37c-93e3-46f8-b9e5-e824b4a369e0'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:51.092688 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-7160d37c-93e3-46f8-b9e5-e824b4a369e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a2ef3b6384a4194a26ba51078c425af', 'request_id': 'req-7160d37c-93e3-46f8-b9e5-e824b4a369e0'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:51.102352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-7160d37c-93e3-46f8-b9e5-e824b4a369e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a2ef3b6384a4194a26ba51078c425af', 'request_id': 'req-7160d37c-93e3-46f8-b9e5-e824b4a369e0'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:51.120520 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-7160d37c-93e3-46f8-b9e5-e824b4a369e0 admin admin] The token cache is being invalidate because project 0a2ef3b6384a4194a26ba51078c425af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:15:51.121038 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-7160d37c-93e3-46f8-b9e5-e824b4a369e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:51.122049 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-7160d37c-93e3-46f8-b9e5-e824b4a369e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:15:51.123652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1229/2481] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:15:50 2022] DELETE /identity/v3/projects/0a2ef3b6384a4194a26ba51078c425af => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:15:53.954791 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141237d-cd97-4612-b134-9c349ec53383 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:53.955382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141237d-cd97-4612-b134-9c349ec53383 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:53.955382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141237d-cd97-4612-b134-9c349ec53383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:54.000398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-e141237d-cd97-4612-b134-9c349ec53383 None None] MFA Rules not processed for user `b3f1ae88e29f4172a7db7393d517a893`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:54.001792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e141237d-cd97-4612-b134-9c349ec53383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:54.132748 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1253/2482] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 23 07:15:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:15:54.150173 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160d37c-93e3-46f8-b9e5-e824b4a369e0 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:15:54.163387 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-7160d37c-93e3-46f8-b9e5-e824b4a369e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:54.328353 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160d37c-93e3-46f8-b9e5-e824b4a369e0 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:15:54.338821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3e5ce-3846-4313-8f20-712611e06d2a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd88ec8b477443af95bff9af03471adf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:15:54.339657 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3e5ce-3846-4313-8f20-712611e06d2a service swift] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:54.339926 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3e5ce-3846-4313-8f20-712611e06d2a service swift] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:54.340200 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3e5ce-3846-4313-8f20-712611e06d2a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:54.353859 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3e5ce-3846-4313-8f20-712611e06d2a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:15:54.358762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3e5ce-3846-4313-8f20-712611e06d2a service swift] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:15:54.375775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1230/2483] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 23 07:15:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 07:15:58.112873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43877f3-c478-4a6e-9e35-784b95657291 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:15:58.113514 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43877f3-c478-4a6e-9e35-784b95657291 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:15:58.113514 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43877f3-c478-4a6e-9e35-784b95657291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:15:58.134789 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-e43877f3-c478-4a6e-9e35-784b95657291 None None] MFA Rules not processed for user `471dc5c1db64411ab1d80ad6c038e519`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:15:58.136067 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e43877f3-c478-4a6e-9e35-784b95657291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:15:58.219205 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1254/2484] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:15:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:16:04.014160 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca5014-e54c-400d-aead-b1b9d690e349 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:04.014725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca5014-e54c-400d-aead-b1b9d690e349 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:04.014835 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca5014-e54c-400d-aead-b1b9d690e349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:04.038299 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-83ca5014-e54c-400d-aead-b1b9d690e349 None None] MFA Rules not processed for user `ffd12f09606b4e6ba12f63b976df1884`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:04.039587 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-83ca5014-e54c-400d-aead-b1b9d690e349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:04.103468 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1231/2485] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:16:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:16:08.202774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=54 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:14.093955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:19.861764 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43877f3-c478-4a6e-9e35-784b95657291 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:19.863306 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e43877f3-c478-4a6e-9e35-784b95657291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:19.927251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43877f3-c478-4a6e-9e35-784b95657291 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:19.931103 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f63e9-a345-462f-9599-a2bc5daf08ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:19.931870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f63e9-a345-462f-9599-a2bc5daf08ca service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:19.932142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f63e9-a345-462f-9599-a2bc5daf08ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:19.932408 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f63e9-a345-462f-9599-a2bc5daf08ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:19.934337 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-032f63e9-a345-462f-9599-a2bc5daf08ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:20.000081 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f63e9-a345-462f-9599-a2bc5daf08ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:20.001421 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f63e9-a345-462f-9599-a2bc5daf08ca service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:20.010949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1255/2486] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:16:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:16:23.311159 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca5014-e54c-400d-aead-b1b9d690e349 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:23.321905 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca5014-e54c-400d-aead-b1b9d690e349 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:23.325517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868bfddc-1ae1-4346-974c-0af4a320067c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:23.326289 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868bfddc-1ae1-4346-974c-0af4a320067c service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:23.326547 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868bfddc-1ae1-4346-974c-0af4a320067c service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:23.328390 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868bfddc-1ae1-4346-974c-0af4a320067c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:23.330298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-868bfddc-1ae1-4346-974c-0af4a320067c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:23.395435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868bfddc-1ae1-4346-974c-0af4a320067c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:23.396898 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868bfddc-1ae1-4346-974c-0af4a320067c service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:23.406671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1232/2487] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:16:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:16:27.562137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f63e9-a345-462f-9599-a2bc5daf08ca service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:16:27.571048 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f63e9-a345-462f-9599-a2bc5daf08ca service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:16:27.575027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ae996-39c1-4d12-83cc-93d53dadf361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:16:27.575657 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ae996-39c1-4d12-83cc-93d53dadf361 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:27.575932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ae996-39c1-4d12-83cc-93d53dadf361 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:27.576247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ae996-39c1-4d12-83cc-93d53dadf361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:27.578399 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-107ae996-39c1-4d12-83cc-93d53dadf361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:27.645163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ae996-39c1-4d12-83cc-93d53dadf361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:16:27.646613 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ae996-39c1-4d12-83cc-93d53dadf361 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:16:27.657038 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1256/2488] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:16:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:16:31.200375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ce406-468f-4d10-8e35-8e5b4d08a0a4 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:16:31.200941 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ce406-468f-4d10-8e35-8e5b4d08a0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:16:31.201068 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ce406-468f-4d10-8e35-8e5b4d08a0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:16:31.222655 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-ad0ce406-468f-4d10-8e35-8e5b4d08a0a4 None None] MFA Rules not processed for user `64d54cb95b264c18b45d90822507fae1`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:16:31.223906 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ad0ce406-468f-4d10-8e35-8e5b4d08a0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:16:31.282268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1233/2489] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:16:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:16:37.652758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=80 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:16:41.274706 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=55 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:17:13.506980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ae996-39c1-4d12-83cc-93d53dadf361 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:13.509028 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-107ae996-39c1-4d12-83cc-93d53dadf361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:13.583850 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ae996-39c1-4d12-83cc-93d53dadf361 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:13.591000 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d1363-7dcc-4b01-a734-82102c9ce412 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:13.591856 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d1363-7dcc-4b01-a734-82102c9ce412 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:13.592130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d1363-7dcc-4b01-a734-82102c9ce412 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:13.592464 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d1363-7dcc-4b01-a734-82102c9ce412 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:13.595589 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-020d1363-7dcc-4b01-a734-82102c9ce412 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:13.685208 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d1363-7dcc-4b01-a734-82102c9ce412 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:13.686673 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d1363-7dcc-4b01-a734-82102c9ce412 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:13.698228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1257/2490] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:17:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:17:15.074280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ce406-468f-4d10-8e35-8e5b4d08a0a4 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:15.076146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ad0ce406-468f-4d10-8e35-8e5b4d08a0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:15.088392 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ce406-468f-4d10-8e35-8e5b4d08a0a4 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:15.091971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc95302-8c92-4da3-a12c-9d2b1afe0fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:15.092582 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc95302-8c92-4da3-a12c-9d2b1afe0fea service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:15.092862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc95302-8c92-4da3-a12c-9d2b1afe0fea service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:15.093137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc95302-8c92-4da3-a12c-9d2b1afe0fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:15.094880 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-cfc95302-8c92-4da3-a12c-9d2b1afe0fea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:15.109466 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc95302-8c92-4da3-a12c-9d2b1afe0fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:15.111399 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc95302-8c92-4da3-a12c-9d2b1afe0fea service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:15.125774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1234/2491] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:17:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:17:20.787840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7480acb-ed6c-4cc7-a88a-395bc2ded19c None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:20.788401 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7480acb-ed6c-4cc7-a88a-395bc2ded19c None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:20.788401 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7480acb-ed6c-4cc7-a88a-395bc2ded19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:20.808117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-a7480acb-ed6c-4cc7-a88a-395bc2ded19c None None] MFA Rules not processed for user `471dc5c1db64411ab1d80ad6c038e519`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:17:20.809324 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a7480acb-ed6c-4cc7-a88a-395bc2ded19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:20.816716 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1258/2492] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:17:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:17:21.480456 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc95302-8c92-4da3-a12c-9d2b1afe0fea service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:21.481900 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-cfc95302-8c92-4da3-a12c-9d2b1afe0fea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:21.561520 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc95302-8c92-4da3-a12c-9d2b1afe0fea service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:21.566516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a673ee0-87b3-45b7-a6ba-d8980fb016eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9b0f54c988e43bda474f978a69fa4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:21.567132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a673ee0-87b3-45b7-a6ba-d8980fb016eb service placement] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:21.567432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a673ee0-87b3-45b7-a6ba-d8980fb016eb service placement] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:21.567746 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a673ee0-87b3-45b7-a6ba-d8980fb016eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:21.569435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3a673ee0-87b3-45b7-a6ba-d8980fb016eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:21.580865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a673ee0-87b3-45b7-a6ba-d8980fb016eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:21.582497 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a673ee0-87b3-45b7-a6ba-d8980fb016eb service placement] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:21.593271 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1235/2493] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:17:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:17:23.889066 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7480acb-ed6c-4cc7-a88a-395bc2ded19c None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:23.891570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a7480acb-ed6c-4cc7-a88a-395bc2ded19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:23.908553 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7480acb-ed6c-4cc7-a88a-395bc2ded19c None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:23.913324 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c25566-0459-4d2a-8ea0-bf4c678f1ff9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:23.914324 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c25566-0459-4d2a-8ea0-bf4c678f1ff9 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:23.914772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c25566-0459-4d2a-8ea0-bf4c678f1ff9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:23.915203 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c25566-0459-4d2a-8ea0-bf4c678f1ff9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:23.918991 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-73c25566-0459-4d2a-8ea0-bf4c678f1ff9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:23.933489 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c25566-0459-4d2a-8ea0-bf4c678f1ff9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:23.934987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c25566-0459-4d2a-8ea0-bf4c678f1ff9 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:23.944781 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1259/2494] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:17:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:17:27.263691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a673ee0-87b3-45b7-a6ba-d8980fb016eb service placement] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:27.265649 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3a673ee0-87b3-45b7-a6ba-d8980fb016eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:27.283852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a673ee0-87b3-45b7-a6ba-d8980fb016eb service placement] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:27.292377 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe321c7-7cd7-4812-8bf0-c8f3e5582597 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:27.293298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe321c7-7cd7-4812-8bf0-c8f3e5582597 service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:27.293588 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe321c7-7cd7-4812-8bf0-c8f3e5582597 service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:27.293922 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe321c7-7cd7-4812-8bf0-c8f3e5582597 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:27.296312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3fe321c7-7cd7-4812-8bf0-c8f3e5582597 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:27.307707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe321c7-7cd7-4812-8bf0-c8f3e5582597 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:27.309328 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe321c7-7cd7-4812-8bf0-c8f3e5582597 service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:27.318706 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1236/2495] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:17:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:17:33.940445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=63 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:17:34.792404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c25566-0459-4d2a-8ea0-bf4c678f1ff9 service cinder] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:34.802553 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c25566-0459-4d2a-8ea0-bf4c678f1ff9 service cinder] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:34.806914 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf34d7-3c52-4919-b9cc-c6f050606d8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:34.807570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf34d7-3c52-4919-b9cc-c6f050606d8e service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:34.807846 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf34d7-3c52-4919-b9cc-c6f050606d8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:34.808117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf34d7-3c52-4919-b9cc-c6f050606d8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:34.810962 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-0edf34d7-3c52-4919-b9cc-c6f050606d8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:34.822315 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf34d7-3c52-4919-b9cc-c6f050606d8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:34.823772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf34d7-3c52-4919-b9cc-c6f050606d8e service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:34.832616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1260/2496] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:17:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:17:37.314746 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=44 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:17:42.910466 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe321c7-7cd7-4812-8bf0-c8f3e5582597 service nova] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:42.912966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3fe321c7-7cd7-4812-8bf0-c8f3e5582597 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:42.929223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe321c7-7cd7-4812-8bf0-c8f3e5582597 service nova] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:42.945812 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ba658-c7a4-4750-bd08-8d03c1da9fb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:42.946795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ba658-c7a4-4750-bd08-8d03c1da9fb8 service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:42.946795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ba658-c7a4-4750-bd08-8d03c1da9fb8 service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:42.947159 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ba658-c7a4-4750-bd08-8d03c1da9fb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:42.949024 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-882ba658-c7a4-4750-bd08-8d03c1da9fb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:42.963460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ba658-c7a4-4750-bd08-8d03c1da9fb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:42.975660 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ba658-c7a4-4750-bd08-8d03c1da9fb8 service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:42.987390 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1237/2497] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:17:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:17:44.030454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf34d7-3c52-4919-b9cc-c6f050606d8e service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:44.039449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf34d7-3c52-4919-b9cc-c6f050606d8e service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:44.043406 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a8878-9cf3-4428-9e54-17b5eacecc2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:44.044056 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a8878-9cf3-4428-9e54-17b5eacecc2d service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:44.044331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a8878-9cf3-4428-9e54-17b5eacecc2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:44.044606 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a8878-9cf3-4428-9e54-17b5eacecc2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:44.046458 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a25a8878-9cf3-4428-9e54-17b5eacecc2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:44.112253 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a8878-9cf3-4428-9e54-17b5eacecc2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:44.113670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a8878-9cf3-4428-9e54-17b5eacecc2d service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:44.123685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1261/2498] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:17:47.399841 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ba658-c7a4-4750-bd08-8d03c1da9fb8 service nova] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:47.418221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ba658-c7a4-4750-bd08-8d03c1da9fb8 service nova] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:47.439075 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebeaeea-5aa5-415a-84aa-dc88b06508ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9b0f54c988e43bda474f978a69fa4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:47.440034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebeaeea-5aa5-415a-84aa-dc88b06508ce service placement] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:47.440908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebeaeea-5aa5-415a-84aa-dc88b06508ce service placement] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:47.443440 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebeaeea-5aa5-415a-84aa-dc88b06508ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:47.453029 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-0ebeaeea-5aa5-415a-84aa-dc88b06508ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:47.470737 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a8878-9cf3-4428-9e54-17b5eacecc2d service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:47.472290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a25a8878-9cf3-4428-9e54-17b5eacecc2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:47.474918 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebeaeea-5aa5-415a-84aa-dc88b06508ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:47.476976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebeaeea-5aa5-415a-84aa-dc88b06508ce service placement] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:47.489122 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1238/2499] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:17:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:17:47.490036 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a8878-9cf3-4428-9e54-17b5eacecc2d service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:47.495925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005358f-765b-419a-ab69-08eaa4b75ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:47.496901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005358f-765b-419a-ab69-08eaa4b75ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:47.497527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005358f-765b-419a-ab69-08eaa4b75ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:47.497999 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005358f-765b-419a-ab69-08eaa4b75ea5 admin admin] PATH_INFO: `/v3/users/64d54cb95b264c18b45d90822507fae1` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:47.499427 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2005358f-765b-419a-ab69-08eaa4b75ea5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64d54cb95b264c18b45d90822507fae1)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:47.502157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2005358f-765b-419a-ab69-08eaa4b75ea5 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:47.562163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-2005358f-765b-419a-ab69-08eaa4b75ea5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64d54cb95b264c18b45d90822507fae1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:17:47.562828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-2005358f-765b-419a-ab69-08eaa4b75ea5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64d54cb95b264c18b45d90822507fae1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:17:47.563530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-2005358f-765b-419a-ab69-08eaa4b75ea5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64d54cb95b264c18b45d90822507fae1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:17:47.570976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-2005358f-765b-419a-ab69-08eaa4b75ea5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64d54cb95b264c18b45d90822507fae1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:17:47.584632 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-2005358f-765b-419a-ab69-08eaa4b75ea5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64d54cb95b264c18b45d90822507fae1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:17:47.598987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1262/2500] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:17:47 2022] DELETE /identity/v3/users/64d54cb95b264c18b45d90822507fae1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:17:47.781820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebeaeea-5aa5-415a-84aa-dc88b06508ce service placement] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:47.793682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebeaeea-5aa5-415a-84aa-dc88b06508ce service placement] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:47.859866 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63e9e8-b3ce-48d9-a7df-1b0c12340d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:47.860534 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63e9e8-b3ce-48d9-a7df-1b0c12340d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:47.860803 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63e9e8-b3ce-48d9-a7df-1b0c12340d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:47.861078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63e9e8-b3ce-48d9-a7df-1b0c12340d15 admin admin] PATH_INFO: `/v3/projects/135367be6e3f4b3d965ee0d99fdca524` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:47.862194 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63e9e8-b3ce-48d9-a7df-1b0c12340d15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=135367be6e3f4b3d965ee0d99fdca524)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:47.864164 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63e9e8-b3ce-48d9-a7df-1b0c12340d15 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:47.932293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-8a63e9e8-b3ce-48d9-a7df-1b0c12340d15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '135367be6e3f4b3d965ee0d99fdca524', 'request_id': 'req-8a63e9e8-b3ce-48d9-a7df-1b0c12340d15'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:17:47.939929 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-8a63e9e8-b3ce-48d9-a7df-1b0c12340d15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '135367be6e3f4b3d965ee0d99fdca524', 'request_id': 'req-8a63e9e8-b3ce-48d9-a7df-1b0c12340d15'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:17:47.948767 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-8a63e9e8-b3ce-48d9-a7df-1b0c12340d15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '135367be6e3f4b3d965ee0d99fdca524', 'request_id': 'req-8a63e9e8-b3ce-48d9-a7df-1b0c12340d15'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:17:47.961803 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-8a63e9e8-b3ce-48d9-a7df-1b0c12340d15 admin admin] The token cache is being invalidate because project 135367be6e3f4b3d965ee0d99fdca524 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:17:47.962086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-8a63e9e8-b3ce-48d9-a7df-1b0c12340d15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:17:47.962743 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-8a63e9e8-b3ce-48d9-a7df-1b0c12340d15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:17:47.964542 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1239/2501] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:17:47 2022] DELETE /identity/v3/projects/135367be6e3f4b3d965ee0d99fdca524 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:17:47.970547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddbb4d-3642-4cb3-b2c4-560c86f4be85 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:47.970833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddbb4d-3642-4cb3-b2c4-560c86f4be85 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:47.971107 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddbb4d-3642-4cb3-b2c4-560c86f4be85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:47.992317 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-0fddbb4d-3642-4cb3-b2c4-560c86f4be85 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:17:47.993524 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-0fddbb4d-3642-4cb3-b2c4-560c86f4be85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:48.043207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:17:48.057871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1263/2502] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:17:48.065901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078802b-6c4d-43a0-8da1-e620ce7624c9 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:48.066199 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078802b-6c4d-43a0-8da1-e620ce7624c9 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:48.066467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078802b-6c4d-43a0-8da1-e620ce7624c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:48.086101 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-0078802b-6c4d-43a0-8da1-e620ce7624c9 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:17:48.087644 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-0078802b-6c4d-43a0-8da1-e620ce7624c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:48.095912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1240/2503] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:17:48.100298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddbb4d-3642-4cb3-b2c4-560c86f4be85 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:48.110671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddbb4d-3642-4cb3-b2c4-560c86f4be85 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:48.116058 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919879cb-23e1-4adb-b393-815b3e3d6274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:48.116681 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919879cb-23e1-4adb-b393-815b3e3d6274 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:48.116955 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919879cb-23e1-4adb-b393-815b3e3d6274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:48.117217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919879cb-23e1-4adb-b393-815b3e3d6274 admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:48.117851 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919879cb-23e1-4adb-b393-815b3e3d6274 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:17:48.118144 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919879cb-23e1-4adb-b393-815b3e3d6274 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:48.119783 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919879cb-23e1-4adb-b393-815b3e3d6274 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:48.132002 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1264/2504] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:17:48.136244 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078802b-6c4d-43a0-8da1-e620ce7624c9 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:48.145944 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078802b-6c4d-43a0-8da1-e620ce7624c9 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:48.150982 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301d3da-82d6-4698-8345-97e6717c1a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:48.151614 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301d3da-82d6-4698-8345-97e6717c1a97 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:48.151892 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301d3da-82d6-4698-8345-97e6717c1a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:48.152187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301d3da-82d6-4698-8345-97e6717c1a97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:48.152931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301d3da-82d6-4698-8345-97e6717c1a97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:48.155001 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301d3da-82d6-4698-8345-97e6717c1a97 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:48.184869 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1241/2505] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:17:48 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:17:48.189088 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919879cb-23e1-4adb-b393-815b3e3d6274 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:48.199657 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919879cb-23e1-4adb-b393-815b3e3d6274 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:48.259153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7cc9b-5a6b-44a2-b854-a87f9d9abdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:48.259783 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7cc9b-5a6b-44a2-b854-a87f9d9abdc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:48.260060 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7cc9b-5a6b-44a2-b854-a87f9d9abdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:48.260333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7cc9b-5a6b-44a2-b854-a87f9d9abdc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:48.260958 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7cc9b-5a6b-44a2-b854-a87f9d9abdc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:48.262900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7cc9b-5a6b-44a2-b854-a87f9d9abdc2 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:48.286661 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1265/2506] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:17:48 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:17:48.291116 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301d3da-82d6-4698-8345-97e6717c1a97 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:48.302318 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301d3da-82d6-4698-8345-97e6717c1a97 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:48.308078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da5634-7fec-48ef-8286-37e1970b763c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:48.308670 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da5634-7fec-48ef-8286-37e1970b763c admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:48.308946 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da5634-7fec-48ef-8286-37e1970b763c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:48.309214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da5634-7fec-48ef-8286-37e1970b763c admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:48.309853 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da5634-7fec-48ef-8286-37e1970b763c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:48.311770 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da5634-7fec-48ef-8286-37e1970b763c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:48.322285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1242/2507] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:17:48.326488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7cc9b-5a6b-44a2-b854-a87f9d9abdc2 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:48.337087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7cc9b-5a6b-44a2-b854-a87f9d9abdc2 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:48.342699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086ef2c-ca0f-4982-8ad3-166ddc673f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:48.343287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086ef2c-ca0f-4982-8ad3-166ddc673f2b admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:48.343598 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086ef2c-ca0f-4982-8ad3-166ddc673f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:48.343867 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086ef2c-ca0f-4982-8ad3-166ddc673f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:48.344510 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086ef2c-ca0f-4982-8ad3-166ddc673f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:48.346097 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086ef2c-ca0f-4982-8ad3-166ddc673f2b admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:48.356978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1266/2508] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:17:48.361238 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da5634-7fec-48ef-8286-37e1970b763c admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:48.371454 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da5634-7fec-48ef-8286-37e1970b763c admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:48.377549 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8871f-bdd9-42d3-b8c0-005cd17c9e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:48.378302 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8871f-bdd9-42d3-b8c0-005cd17c9e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:48.378613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8871f-bdd9-42d3-b8c0-005cd17c9e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:48.378891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8871f-bdd9-42d3-b8c0-005cd17c9e90 admin admin] PATH_INFO: `/v3/projects/42af08ef367945939b1f59ffa8af351a/users/4fb28547f4b54c4097d2ffc3c533ca67/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:48.394630 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb8871f-bdd9-42d3-b8c0-005cd17c9e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42af08ef367945939b1f59ffa8af351a, user_id=4fb28547f4b54c4097d2ffc3c533ca67, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:48.397396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb8871f-bdd9-42d3-b8c0-005cd17c9e90 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:48.407681 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:17:48.412709 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1243/2509] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:17:48 2022] PUT /identity/v3/projects/42af08ef367945939b1f59ffa8af351a/users/4fb28547f4b54c4097d2ffc3c533ca67/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:17:48.418206 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2eb3d-0a89-46a9-b231-39fa4127e7e0 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:48.418492 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2eb3d-0a89-46a9-b231-39fa4127e7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:48.418767 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2eb3d-0a89-46a9-b231-39fa4127e7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:48.437725 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-23a2eb3d-0a89-46a9-b231-39fa4127e7e0 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:17:48.438965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-23a2eb3d-0a89-46a9-b231-39fa4127e7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:48.509335 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1267/2510] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:17:48.519261 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8871f-bdd9-42d3-b8c0-005cd17c9e90 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:48.521324 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9fb8871f-bdd9-42d3-b8c0-005cd17c9e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:48.583487 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8871f-bdd9-42d3-b8c0-005cd17c9e90 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:48.588092 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da05b1-1ce3-4abc-9010-12dfe2aa6f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:48.588699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da05b1-1ce3-4abc-9010-12dfe2aa6f04 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:48.588970 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da05b1-1ce3-4abc-9010-12dfe2aa6f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:48.589236 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da05b1-1ce3-4abc-9010-12dfe2aa6f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:48.599259 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da05b1-1ce3-4abc-9010-12dfe2aa6f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:48.600748 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da05b1-1ce3-4abc-9010-12dfe2aa6f04 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:48.610565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1244/2511] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:17:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:17:52.533801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47fb8c-8dc3-42f1-86be-9c17709bada3 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:52.534290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47fb8c-8dc3-42f1-86be-9c17709bada3 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:52.535293 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47fb8c-8dc3-42f1-86be-9c17709bada3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:52.573031 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-1b47fb8c-8dc3-42f1-86be-9c17709bada3 None None] MFA Rules not processed for user `4fb28547f4b54c4097d2ffc3c533ca67`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:17:52.574373 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-1b47fb8c-8dc3-42f1-86be-9c17709bada3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:52.598725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da05b1-1ce3-4abc-9010-12dfe2aa6f04 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:52.600453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-00da05b1-1ce3-4abc-9010-12dfe2aa6f04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:52.648183 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:17:52.695615 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1268/2512] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:17:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:17:52.702879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47fb8c-8dc3-42f1-86be-9c17709bada3 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:52.704745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-1b47fb8c-8dc3-42f1-86be-9c17709bada3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:52.745164 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da05b1-1ce3-4abc-9010-12dfe2aa6f04 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:52.757648 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e5d8c-93ff-4efc-b95d-c4e1ec174db8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:52.758214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e5d8c-93ff-4efc-b95d-c4e1ec174db8 service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:52.758491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e5d8c-93ff-4efc-b95d-c4e1ec174db8 service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:52.758757 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e5d8c-93ff-4efc-b95d-c4e1ec174db8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:52.761129 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b55e5d8c-93ff-4efc-b95d-c4e1ec174db8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:52.781587 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e5d8c-93ff-4efc-b95d-c4e1ec174db8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:52.784672 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e5d8c-93ff-4efc-b95d-c4e1ec174db8 service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:52.802872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1245/2513] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:17:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:17:52.857551 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47fb8c-8dc3-42f1-86be-9c17709bada3 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:52.870749 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffadcd6-ebcc-400f-af2c-54754dcb5115 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:52.871195 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffadcd6-ebcc-400f-af2c-54754dcb5115 service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:52.871497 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffadcd6-ebcc-400f-af2c-54754dcb5115 service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:52.871766 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffadcd6-ebcc-400f-af2c-54754dcb5115 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:52.894308 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffadcd6-ebcc-400f-af2c-54754dcb5115 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:52.896630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffadcd6-ebcc-400f-af2c-54754dcb5115 service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:52.906934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1269/2514] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:17:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:17:54.695149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f16cc-d7ec-4b9f-97f0-f9c7b8017d54 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:54.695707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f16cc-d7ec-4b9f-97f0-f9c7b8017d54 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:54.695707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f16cc-d7ec-4b9f-97f0-f9c7b8017d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:54.716212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-214f16cc-d7ec-4b9f-97f0-f9c7b8017d54 None None] MFA Rules not processed for user `4fb28547f4b54c4097d2ffc3c533ca67`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:17:54.717771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-214f16cc-d7ec-4b9f-97f0-f9c7b8017d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:54.724924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1246/2515] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:17:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:17:55.091721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffadcd6-ebcc-400f-af2c-54754dcb5115 service nova] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:17:55.094114 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-2ffadcd6-ebcc-400f-af2c-54754dcb5115 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:55.123557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffadcd6-ebcc-400f-af2c-54754dcb5115 service nova] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:17:55.131745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e57d6e-16e3-4dcf-be66-4f3d2e983091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:17:55.132377 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e57d6e-16e3-4dcf-be66-4f3d2e983091 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:17:55.132657 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e57d6e-16e3-4dcf-be66-4f3d2e983091 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:17:55.132931 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e57d6e-16e3-4dcf-be66-4f3d2e983091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:17:55.134913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-63e57d6e-16e3-4dcf-be66-4f3d2e983091 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:17:55.145646 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e57d6e-16e3-4dcf-be66-4f3d2e983091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:17:55.147357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e57d6e-16e3-4dcf-be66-4f3d2e983091 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:17:55.156510 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1270/2516] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:17:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:18:00.775068 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59223ec-fc4b-41a4-a250-c0464b8d6e95 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:00.775539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59223ec-fc4b-41a4-a250-c0464b8d6e95 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:00.775657 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59223ec-fc4b-41a4-a250-c0464b8d6e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:00.800297 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-d59223ec-fc4b-41a4-a250-c0464b8d6e95 None None] MFA Rules not processed for user `b3f1ae88e29f4172a7db7393d517a893`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:00.802015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d59223ec-fc4b-41a4-a250-c0464b8d6e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:00.862142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1247/2517] 10.222.0.17 () {58 vars in 967 bytes} [Tue Aug 23 07:18:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:00.903105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e57d6e-16e3-4dcf-be66-4f3d2e983091 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:00.904635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-63e57d6e-16e3-4dcf-be66-4f3d2e983091 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:00.976962 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e57d6e-16e3-4dcf-be66-4f3d2e983091 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:00.982411 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5e658-8ba2-4018-9062-6f6236c51542 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd88ec8b477443af95bff9af03471adf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:00.983015 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5e658-8ba2-4018-9062-6f6236c51542 service swift] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:00.983289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5e658-8ba2-4018-9062-6f6236c51542 service swift] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:00.983592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5e658-8ba2-4018-9062-6f6236c51542 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:00.992261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5e658-8ba2-4018-9062-6f6236c51542 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:00.993843 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5e658-8ba2-4018-9062-6f6236c51542 service swift] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:01.002420 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1271/2518] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 23 07:18:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 07:18:04.864574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59223ec-fc4b-41a4-a250-c0464b8d6e95 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:04.878465 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59223ec-fc4b-41a4-a250-c0464b8d6e95 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:04.892396 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c546a8-c3ee-420b-8473-2548db277055 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:04.893019 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c546a8-c3ee-420b-8473-2548db277055 service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:04.893301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c546a8-c3ee-420b-8473-2548db277055 service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:04.893575 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c546a8-c3ee-420b-8473-2548db277055 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:04.900706 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-05c546a8-c3ee-420b-8473-2548db277055 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:04.926034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c546a8-c3ee-420b-8473-2548db277055 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:04.928267 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c546a8-c3ee-420b-8473-2548db277055 service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:04.949252 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1248/2519] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:18:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:18:08.343001 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5e658-8ba2-4018-9062-6f6236c51542 service swift] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:08.344521 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ece5e658-8ba2-4018-9062-6f6236c51542 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:08.358858 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5e658-8ba2-4018-9062-6f6236c51542 service swift] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:08.364671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfd517-9434-4394-9394-121fad19aac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:08.365483 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfd517-9434-4394-9394-121fad19aac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:08.365918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfd517-9434-4394-9394-121fad19aac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:08.366384 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfd517-9434-4394-9394-121fad19aac2 admin admin] PATH_INFO: `/v3/users/a5e8c33634b246319de495d269c05124` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:08.370056 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfd517-9434-4394-9394-121fad19aac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5e8c33634b246319de495d269c05124)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:08.372045 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfd517-9434-4394-9394-121fad19aac2 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:08.424482 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:08.426389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-42dfd517-9434-4394-9394-121fad19aac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5e8c33634b246319de495d269c05124'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:08.427338 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-42dfd517-9434-4394-9394-121fad19aac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5e8c33634b246319de495d269c05124'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:08.436059 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-42dfd517-9434-4394-9394-121fad19aac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5e8c33634b246319de495d269c05124'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:08.444198 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-42dfd517-9434-4394-9394-121fad19aac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5e8c33634b246319de495d269c05124'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:08.457257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-42dfd517-9434-4394-9394-121fad19aac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5e8c33634b246319de495d269c05124'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:08.471690 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1272/2520] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:18:08 2022] DELETE /identity/v3/users/a5e8c33634b246319de495d269c05124 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:08.645565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c546a8-c3ee-420b-8473-2548db277055 service nova] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:08.660781 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c546a8-c3ee-420b-8473-2548db277055 service nova] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:08.683208 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:08.735245 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec8700-fcf3-4baa-ada3-15c0eff7d63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:08.735931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec8700-fcf3-4baa-ada3-15c0eff7d63e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:08.736214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec8700-fcf3-4baa-ada3-15c0eff7d63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:08.736514 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec8700-fcf3-4baa-ada3-15c0eff7d63e admin admin] PATH_INFO: `/v3/projects/f9466597900d4a3ca60a2ef8b7678c87` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:08.737661 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec8700-fcf3-4baa-ada3-15c0eff7d63e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9466597900d4a3ca60a2ef8b7678c87)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:08.739755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec8700-fcf3-4baa-ada3-15c0eff7d63e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:08.792260 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-eeec8700-fcf3-4baa-ada3-15c0eff7d63e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9466597900d4a3ca60a2ef8b7678c87', 'request_id': 'req-eeec8700-fcf3-4baa-ada3-15c0eff7d63e'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:08.798621 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-eeec8700-fcf3-4baa-ada3-15c0eff7d63e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9466597900d4a3ca60a2ef8b7678c87', 'request_id': 'req-eeec8700-fcf3-4baa-ada3-15c0eff7d63e'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:08.806880 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-eeec8700-fcf3-4baa-ada3-15c0eff7d63e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9466597900d4a3ca60a2ef8b7678c87', 'request_id': 'req-eeec8700-fcf3-4baa-ada3-15c0eff7d63e'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:08.820473 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-eeec8700-fcf3-4baa-ada3-15c0eff7d63e admin admin] The token cache is being invalidate because project f9466597900d4a3ca60a2ef8b7678c87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:18:08.820733 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-eeec8700-fcf3-4baa-ada3-15c0eff7d63e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:08.821532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-eeec8700-fcf3-4baa-ada3-15c0eff7d63e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:08.823629 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1249/2521] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:18:08 2022] DELETE /identity/v3/projects/f9466597900d4a3ca60a2ef8b7678c87 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:08.828061 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfd517-9434-4394-9394-121fad19aac2 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:08.829720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-42dfd517-9434-4394-9394-121fad19aac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:08.844479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfd517-9434-4394-9394-121fad19aac2 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:08.905570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f731b31-3d98-491a-822d-a4a7d2296274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:08.906207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f731b31-3d98-491a-822d-a4a7d2296274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:08.906485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f731b31-3d98-491a-822d-a4a7d2296274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:08.906760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f731b31-3d98-491a-822d-a4a7d2296274 admin admin] PATH_INFO: `/v3/users/3751f05926f7439984f0c564d20b4086` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:08.907954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f731b31-3d98-491a-822d-a4a7d2296274 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3751f05926f7439984f0c564d20b4086)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:08.909883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f731b31-3d98-491a-822d-a4a7d2296274 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:08.960026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-3f731b31-3d98-491a-822d-a4a7d2296274 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3751f05926f7439984f0c564d20b4086'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:08.960720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-3f731b31-3d98-491a-822d-a4a7d2296274 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3751f05926f7439984f0c564d20b4086'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:08.961376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-3f731b31-3d98-491a-822d-a4a7d2296274 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3751f05926f7439984f0c564d20b4086'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:08.968966 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-3f731b31-3d98-491a-822d-a4a7d2296274 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3751f05926f7439984f0c564d20b4086'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:08.981320 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-3f731b31-3d98-491a-822d-a4a7d2296274 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3751f05926f7439984f0c564d20b4086'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:08.997485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1273/2522] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:18:08 2022] DELETE /identity/v3/users/3751f05926f7439984f0c564d20b4086 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:18:09.177594 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec8700-fcf3-4baa-ada3-15c0eff7d63e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:09.184332 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-eeec8700-fcf3-4baa-ada3-15c0eff7d63e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:09.211463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec8700-fcf3-4baa-ada3-15c0eff7d63e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:09.271928 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a430d-e7a7-41f1-a4f0-306237875250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:09.272565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a430d-e7a7-41f1-a4f0-306237875250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:09.272843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a430d-e7a7-41f1-a4f0-306237875250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:09.273116 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a430d-e7a7-41f1-a4f0-306237875250 admin admin] PATH_INFO: `/v3/projects/e6b6e9084ed6444aa677efceac802ec0` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:09.274235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a430d-e7a7-41f1-a4f0-306237875250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6b6e9084ed6444aa677efceac802ec0)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:09.276337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a430d-e7a7-41f1-a4f0-306237875250 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:09.323877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:09.326572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-fe4a430d-e7a7-41f1-a4f0-306237875250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6b6e9084ed6444aa677efceac802ec0', 'request_id': 'req-fe4a430d-e7a7-41f1-a4f0-306237875250'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:09.334719 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-fe4a430d-e7a7-41f1-a4f0-306237875250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6b6e9084ed6444aa677efceac802ec0', 'request_id': 'req-fe4a430d-e7a7-41f1-a4f0-306237875250'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:09.345221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-fe4a430d-e7a7-41f1-a4f0-306237875250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6b6e9084ed6444aa677efceac802ec0', 'request_id': 'req-fe4a430d-e7a7-41f1-a4f0-306237875250'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:09.356971 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-fe4a430d-e7a7-41f1-a4f0-306237875250 admin admin] The token cache is being invalidate because project e6b6e9084ed6444aa677efceac802ec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:18:09.357273 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-fe4a430d-e7a7-41f1-a4f0-306237875250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:09.357911 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-fe4a430d-e7a7-41f1-a4f0-306237875250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:09.359439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1250/2523] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:18:09 2022] DELETE /identity/v3/projects/e6b6e9084ed6444aa677efceac802ec0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:18.993622 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=66,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:19.353878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:28.509611 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f731b31-3d98-491a-822d-a4a7d2296274 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:28.511182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3f731b31-3d98-491a-822d-a4a7d2296274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:28.579465 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f731b31-3d98-491a-822d-a4a7d2296274 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:28.635544 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe6642-037b-45d8-bc0a-c6c9f22c934e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:28.636188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe6642-037b-45d8-bc0a-c6c9f22c934e service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:28.636552 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe6642-037b-45d8-bc0a-c6c9f22c934e service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:28.636908 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe6642-037b-45d8-bc0a-c6c9f22c934e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:28.639515 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-92fe6642-037b-45d8-bc0a-c6c9f22c934e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:28.702948 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe6642-037b-45d8-bc0a-c6c9f22c934e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:28.704420 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe6642-037b-45d8-bc0a-c6c9f22c934e service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:28.714345 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1274/2524] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:18:31.678372 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a430d-e7a7-41f1-a4f0-306237875250 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:31.680036 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-fe4a430d-e7a7-41f1-a4f0-306237875250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:31.743325 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe6642-037b-45d8-bc0a-c6c9f22c934e service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:31.745052 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-92fe6642-037b-45d8-bc0a-c6c9f22c934e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:31.759118 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe6642-037b-45d8-bc0a-c6c9f22c934e service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:31.764180 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a430d-e7a7-41f1-a4f0-306237875250 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:31.768783 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52216820-7e21-4831-bb09-6a58729e2715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:31.769626 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52216820-7e21-4831-bb09-6a58729e2715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:31.770066 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52216820-7e21-4831-bb09-6a58729e2715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:31.770336 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52216820-7e21-4831-bb09-6a58729e2715 admin admin] PATH_INFO: `/v3/users/471dc5c1db64411ab1d80ad6c038e519` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:31.771369 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f50f32-d7d2-4dea-82f0-73c046656491 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9b0f54c988e43bda474f978a69fa4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:31.771580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52216820-7e21-4831-bb09-6a58729e2715 admin admin] RBAC: Authorizing `identity:delete_user(user_id=471dc5c1db64411ab1d80ad6c038e519)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:31.772163 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f50f32-d7d2-4dea-82f0-73c046656491 service placement] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:31.772426 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f50f32-d7d2-4dea-82f0-73c046656491 service placement] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:31.772700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f50f32-d7d2-4dea-82f0-73c046656491 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:31.773526 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52216820-7e21-4831-bb09-6a58729e2715 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:31.775136 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-73f50f32-d7d2-4dea-82f0-73c046656491 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:31.787552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f50f32-d7d2-4dea-82f0-73c046656491 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:31.789170 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f50f32-d7d2-4dea-82f0-73c046656491 service placement] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:31.800372 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1251/2525] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:18:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:18:31.829405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-52216820-7e21-4831-bb09-6a58729e2715 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '471dc5c1db64411ab1d80ad6c038e519'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:31.830073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-52216820-7e21-4831-bb09-6a58729e2715 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '471dc5c1db64411ab1d80ad6c038e519'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:31.830828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-52216820-7e21-4831-bb09-6a58729e2715 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '471dc5c1db64411ab1d80ad6c038e519'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:31.841911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-52216820-7e21-4831-bb09-6a58729e2715 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '471dc5c1db64411ab1d80ad6c038e519'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:31.846936 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:31.857767 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-52216820-7e21-4831-bb09-6a58729e2715 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '471dc5c1db64411ab1d80ad6c038e519'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:31.873311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1275/2526] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:18:31 2022] DELETE /identity/v3/users/471dc5c1db64411ab1d80ad6c038e519 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:18:32.054631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f50f32-d7d2-4dea-82f0-73c046656491 service placement] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:32.069749 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f50f32-d7d2-4dea-82f0-73c046656491 service placement] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:32.134544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a98a91-460d-48c8-9afb-dcab1a0f7072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:32.135190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a98a91-460d-48c8-9afb-dcab1a0f7072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:32.135498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a98a91-460d-48c8-9afb-dcab1a0f7072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:32.135773 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a98a91-460d-48c8-9afb-dcab1a0f7072 admin admin] PATH_INFO: `/v3/projects/eb44f62fa62e46528712f24479cbb19f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:32.136909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a98a91-460d-48c8-9afb-dcab1a0f7072 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb44f62fa62e46528712f24479cbb19f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:32.138928 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a98a91-460d-48c8-9afb-dcab1a0f7072 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:32.189726 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-73a98a91-460d-48c8-9afb-dcab1a0f7072 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb44f62fa62e46528712f24479cbb19f', 'request_id': 'req-73a98a91-460d-48c8-9afb-dcab1a0f7072'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:32.195986 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-73a98a91-460d-48c8-9afb-dcab1a0f7072 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb44f62fa62e46528712f24479cbb19f', 'request_id': 'req-73a98a91-460d-48c8-9afb-dcab1a0f7072'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:32.203752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-73a98a91-460d-48c8-9afb-dcab1a0f7072 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb44f62fa62e46528712f24479cbb19f', 'request_id': 'req-73a98a91-460d-48c8-9afb-dcab1a0f7072'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:32.215849 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-73a98a91-460d-48c8-9afb-dcab1a0f7072 admin admin] The token cache is being invalidate because project eb44f62fa62e46528712f24479cbb19f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:18:32.216184 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-73a98a91-460d-48c8-9afb-dcab1a0f7072 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:32.216888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-73a98a91-460d-48c8-9afb-dcab1a0f7072 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:32.218560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1252/2527] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:18:32 2022] DELETE /identity/v3/projects/eb44f62fa62e46528712f24479cbb19f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:32.223574 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf17eba-28ed-49d2-a291-511e24f64a63 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:32.224069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf17eba-28ed-49d2-a291-511e24f64a63 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:32.224542 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf17eba-28ed-49d2-a291-511e24f64a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:32.245736 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-dbf17eba-28ed-49d2-a291-511e24f64a63 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:32.246959 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-dbf17eba-28ed-49d2-a291-511e24f64a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:32.308590 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1276/2528] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:32.317685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99194ce1-83fd-4926-8002-793b0d2f037b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:32.317995 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99194ce1-83fd-4926-8002-793b0d2f037b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:32.318252 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99194ce1-83fd-4926-8002-793b0d2f037b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:32.340553 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-99194ce1-83fd-4926-8002-793b0d2f037b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:32.341894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-99194ce1-83fd-4926-8002-793b0d2f037b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:32.351145 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1253/2529] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:18:32.355523 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf17eba-28ed-49d2-a291-511e24f64a63 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:32.365793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf17eba-28ed-49d2-a291-511e24f64a63 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:32.373578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e2698-8674-4ff8-9213-8032d2ada483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:32.374408 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e2698-8674-4ff8-9213-8032d2ada483 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:32.374735 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e2698-8674-4ff8-9213-8032d2ada483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:32.375036 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e2698-8674-4ff8-9213-8032d2ada483 admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:32.376065 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e2698-8674-4ff8-9213-8032d2ada483 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:32.376486 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e2698-8674-4ff8-9213-8032d2ada483 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:32.379957 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e2698-8674-4ff8-9213-8032d2ada483 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:32.399145 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1277/2530] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:18:32.405911 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99194ce1-83fd-4926-8002-793b0d2f037b None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:32.415978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99194ce1-83fd-4926-8002-793b0d2f037b None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:32.421760 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6156ff43-f6af-4367-b9aa-1d42bec4b442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:32.422367 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156ff43-f6af-4367-b9aa-1d42bec4b442 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:32.422644 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156ff43-f6af-4367-b9aa-1d42bec4b442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:32.422917 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156ff43-f6af-4367-b9aa-1d42bec4b442 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:32.423622 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6156ff43-f6af-4367-b9aa-1d42bec4b442 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:32.425613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6156ff43-f6af-4367-b9aa-1d42bec4b442 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:32.437570 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:32.458060 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1254/2531] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:18:32 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:32.462886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e2698-8674-4ff8-9213-8032d2ada483 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:32.473089 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e2698-8674-4ff8-9213-8032d2ada483 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:32.531854 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119f3ec-df03-4ddd-8ff5-23fc695e9b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:32.532449 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119f3ec-df03-4ddd-8ff5-23fc695e9b93 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:32.532722 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119f3ec-df03-4ddd-8ff5-23fc695e9b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:32.532989 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119f3ec-df03-4ddd-8ff5-23fc695e9b93 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:32.533609 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8119f3ec-df03-4ddd-8ff5-23fc695e9b93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:32.535563 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8119f3ec-df03-4ddd-8ff5-23fc695e9b93 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:32.559264 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1278/2532] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:18:32 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:32.563688 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6156ff43-f6af-4367-b9aa-1d42bec4b442 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:32.574472 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6156ff43-f6af-4367-b9aa-1d42bec4b442 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:32.580489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5c6d5-0e04-40b2-a793-8342f7eebb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:32.581140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5c6d5-0e04-40b2-a793-8342f7eebb5b admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:32.581421 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5c6d5-0e04-40b2-a793-8342f7eebb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:32.581693 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5c6d5-0e04-40b2-a793-8342f7eebb5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:32.582342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5c6d5-0e04-40b2-a793-8342f7eebb5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:32.584149 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5c6d5-0e04-40b2-a793-8342f7eebb5b admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:32.595326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1255/2533] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:32.599687 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119f3ec-df03-4ddd-8ff5-23fc695e9b93 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:32.610742 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119f3ec-df03-4ddd-8ff5-23fc695e9b93 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:32.616823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be59a4-6ae7-4d9a-b036-8549f7bcb448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:32.617477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be59a4-6ae7-4d9a-b036-8549f7bcb448 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:32.617751 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be59a4-6ae7-4d9a-b036-8549f7bcb448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:32.618019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be59a4-6ae7-4d9a-b036-8549f7bcb448 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:32.618651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be59a4-6ae7-4d9a-b036-8549f7bcb448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:32.620371 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be59a4-6ae7-4d9a-b036-8549f7bcb448 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:32.632131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1279/2534] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:18:32.636477 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5c6d5-0e04-40b2-a793-8342f7eebb5b admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:32.646881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5c6d5-0e04-40b2-a793-8342f7eebb5b admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:32.652334 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ec5bc-acea-4880-9f2d-4c8dbf2ef9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:32.652973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ec5bc-acea-4880-9f2d-4c8dbf2ef9b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:32.653255 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ec5bc-acea-4880-9f2d-4c8dbf2ef9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:32.653534 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ec5bc-acea-4880-9f2d-4c8dbf2ef9b1 admin admin] PATH_INFO: `/v3/projects/d04597cbe7ce4eb6b187949526c924ce/users/7d15c2bfc8cd483cadd3fc941804e5f7/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:32.670696 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ec5bc-acea-4880-9f2d-4c8dbf2ef9b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d04597cbe7ce4eb6b187949526c924ce, user_id=7d15c2bfc8cd483cadd3fc941804e5f7, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:32.673417 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ec5bc-acea-4880-9f2d-4c8dbf2ef9b1 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:32.689652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1256/2535] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:32 2022] PUT /identity/v3/projects/d04597cbe7ce4eb6b187949526c924ce/users/7d15c2bfc8cd483cadd3fc941804e5f7/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:32.705790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43888770-0453-4517-b5ce-bcc0f313a735 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:32.706217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43888770-0453-4517-b5ce-bcc0f313a735 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:32.706639 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43888770-0453-4517-b5ce-bcc0f313a735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:32.738335 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-43888770-0453-4517-b5ce-bcc0f313a735 None None] MFA Rules not processed for user `7d15c2bfc8cd483cadd3fc941804e5f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:32.739575 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-43888770-0453-4517-b5ce-bcc0f313a735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:32.775040 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:32.809974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1280/2536] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:32.814822 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ec5bc-acea-4880-9f2d-4c8dbf2ef9b1 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:32.884511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ec5bc-acea-4880-9f2d-4c8dbf2ef9b1 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:32.890629 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b9bc7-3e31-4374-972d-ea00e5d3b3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:32.891269 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b9bc7-3e31-4374-972d-ea00e5d3b3e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:32.891577 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b9bc7-3e31-4374-972d-ea00e5d3b3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:32.891850 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b9bc7-3e31-4374-972d-ea00e5d3b3e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:32.892471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b9bc7-3e31-4374-972d-ea00e5d3b3e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:32.894398 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b9bc7-3e31-4374-972d-ea00e5d3b3e9 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:32.924150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1257/2537] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:18:32 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:32.928485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43888770-0453-4517-b5ce-bcc0f313a735 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:32.939770 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43888770-0453-4517-b5ce-bcc0f313a735 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:33.000075 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4730b-4b7a-4b1a-883d-8821cd98be96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:33.000681 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4730b-4b7a-4b1a-883d-8821cd98be96 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:33.000962 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4730b-4b7a-4b1a-883d-8821cd98be96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:33.001236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4730b-4b7a-4b1a-883d-8821cd98be96 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:33.001868 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4730b-4b7a-4b1a-883d-8821cd98be96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:33.003873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4730b-4b7a-4b1a-883d-8821cd98be96 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:33.030463 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1281/2538] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:18:32 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:33.035617 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b9bc7-3e31-4374-972d-ea00e5d3b3e9 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:33.050555 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b9bc7-3e31-4374-972d-ea00e5d3b3e9 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:33.056381 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8d871-3ce7-4a0a-81d7-127c53c3dd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:33.057007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8d871-3ce7-4a0a-81d7-127c53c3dd0a admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:33.057277 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8d871-3ce7-4a0a-81d7-127c53c3dd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:33.057550 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8d871-3ce7-4a0a-81d7-127c53c3dd0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:33.058191 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d8d871-3ce7-4a0a-81d7-127c53c3dd0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:33.059832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d8d871-3ce7-4a0a-81d7-127c53c3dd0a admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:33.072693 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1258/2539] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:33.077186 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4730b-4b7a-4b1a-883d-8821cd98be96 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:33.091040 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4730b-4b7a-4b1a-883d-8821cd98be96 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:33.096437 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b98986-717c-4b56-952c-0dfd3c1841a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:33.097078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b98986-717c-4b56-952c-0dfd3c1841a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:33.097354 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b98986-717c-4b56-952c-0dfd3c1841a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:33.097592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b98986-717c-4b56-952c-0dfd3c1841a3 admin admin] PATH_INFO: `/v3/projects/fcf026244e374b9198f1f75e6ee2ed99/users/ec55bf68cafd4ccbae758c899a685ec6/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:33.114687 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b98986-717c-4b56-952c-0dfd3c1841a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcf026244e374b9198f1f75e6ee2ed99, user_id=ec55bf68cafd4ccbae758c899a685ec6, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:33.117527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b98986-717c-4b56-952c-0dfd3c1841a3 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:33.133943 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1282/2540] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:33 2022] PUT /identity/v3/projects/fcf026244e374b9198f1f75e6ee2ed99/users/ec55bf68cafd4ccbae758c899a685ec6/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:33.138243 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8d871-3ce7-4a0a-81d7-127c53c3dd0a admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:33.148633 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8d871-3ce7-4a0a-81d7-127c53c3dd0a admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:33.207948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66fa7a-7f2a-4c0e-922e-5127937f14a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:33.208464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66fa7a-7f2a-4c0e-922e-5127937f14a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:33.208740 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66fa7a-7f2a-4c0e-922e-5127937f14a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:33.209007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66fa7a-7f2a-4c0e-922e-5127937f14a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:33.209632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66fa7a-7f2a-4c0e-922e-5127937f14a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:33.211257 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66fa7a-7f2a-4c0e-922e-5127937f14a0 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:33.221484 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1259/2541] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:33.227119 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b98986-717c-4b56-952c-0dfd3c1841a3 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:33.237817 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b98986-717c-4b56-952c-0dfd3c1841a3 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:33.243006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e655900-9256-46be-ae2d-61f70e0f3828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:33.243793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e655900-9256-46be-ae2d-61f70e0f3828 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:33.244069 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e655900-9256-46be-ae2d-61f70e0f3828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:33.244341 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e655900-9256-46be-ae2d-61f70e0f3828 admin admin] PATH_INFO: `/v3/projects/fcf026244e374b9198f1f75e6ee2ed99/users/ec55bf68cafd4ccbae758c899a685ec6/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:33.247102 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e655900-9256-46be-ae2d-61f70e0f3828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcf026244e374b9198f1f75e6ee2ed99, user_id=ec55bf68cafd4ccbae758c899a685ec6, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:33.250555 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e655900-9256-46be-ae2d-61f70e0f3828 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:33.262354 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1283/2542] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:33 2022] PUT /identity/v3/projects/fcf026244e374b9198f1f75e6ee2ed99/users/ec55bf68cafd4ccbae758c899a685ec6/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:33.273258 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729fd81-c7a0-4554-9daa-8c6c11028171 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:33.273534 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729fd81-c7a0-4554-9daa-8c6c11028171 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:33.273824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729fd81-c7a0-4554-9daa-8c6c11028171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:33.302529 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-1729fd81-c7a0-4554-9daa-8c6c11028171 None None] MFA Rules not processed for user `ec55bf68cafd4ccbae758c899a685ec6`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:33.303749 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-1729fd81-c7a0-4554-9daa-8c6c11028171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:33.317135 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:33.378073 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1260/2543] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:33.389814 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e655900-9256-46be-ae2d-61f70e0f3828 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:33.391734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-2e655900-9256-46be-ae2d-61f70e0f3828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:33.465793 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e655900-9256-46be-ae2d-61f70e0f3828 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:33.470285 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5170f-e293-47cd-b8ff-4e3bfa7c2c0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:33.470894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5170f-e293-47cd-b8ff-4e3bfa7c2c0e service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:33.471171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5170f-e293-47cd-b8ff-4e3bfa7c2c0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:33.471483 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5170f-e293-47cd-b8ff-4e3bfa7c2c0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:33.480546 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5170f-e293-47cd-b8ff-4e3bfa7c2c0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:33.482001 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5170f-e293-47cd-b8ff-4e3bfa7c2c0e service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:33.492941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1284/2544] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:18:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:18:33.973056 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729fd81-c7a0-4554-9daa-8c6c11028171 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:34.038480 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729fd81-c7a0-4554-9daa-8c6c11028171 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:34.043833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f2dcb-018f-457e-a7d4-73850177eac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:34.044481 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f2dcb-018f-457e-a7d4-73850177eac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:34.044755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f2dcb-018f-457e-a7d4-73850177eac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:34.045025 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f2dcb-018f-457e-a7d4-73850177eac0 admin admin] PATH_INFO: `/v3/users/7d15c2bfc8cd483cadd3fc941804e5f7` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:34.046228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f2dcb-018f-457e-a7d4-73850177eac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d15c2bfc8cd483cadd3fc941804e5f7)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:34.048449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f2dcb-018f-457e-a7d4-73850177eac0 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:34.097160 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-859f2dcb-018f-457e-a7d4-73850177eac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d15c2bfc8cd483cadd3fc941804e5f7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:34.097848 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-859f2dcb-018f-457e-a7d4-73850177eac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d15c2bfc8cd483cadd3fc941804e5f7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:34.098462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-859f2dcb-018f-457e-a7d4-73850177eac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d15c2bfc8cd483cadd3fc941804e5f7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:34.105244 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-859f2dcb-018f-457e-a7d4-73850177eac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d15c2bfc8cd483cadd3fc941804e5f7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:34.121552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-859f2dcb-018f-457e-a7d4-73850177eac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d15c2bfc8cd483cadd3fc941804e5f7'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:34.139230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1261/2545] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:18:33 2022] DELETE /identity/v3/users/7d15c2bfc8cd483cadd3fc941804e5f7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:34.144640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fa75c-da7e-4096-a36c-a416a024a0d4 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:34.145163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fa75c-da7e-4096-a36c-a416a024a0d4 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:34.145163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fa75c-da7e-4096-a36c-a416a024a0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:34.165015 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-699fa75c-da7e-4096-a36c-a416a024a0d4 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:34.166233 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-699fa75c-da7e-4096-a36c-a416a024a0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:34.192525 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:34.229232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1285/2546] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:34.245518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f2dcb-018f-457e-a7d4-73850177eac0 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:34.248064 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-859f2dcb-018f-457e-a7d4-73850177eac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:34.288112 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:34.314592 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f2dcb-018f-457e-a7d4-73850177eac0 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:34.318609 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb826c2-ebab-4344-addb-18b2dcd4c06d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:34.319212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb826c2-ebab-4344-addb-18b2dcd4c06d service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:34.319545 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb826c2-ebab-4344-addb-18b2dcd4c06d service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:34.319815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb826c2-ebab-4344-addb-18b2dcd4c06d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:34.329753 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb826c2-ebab-4344-addb-18b2dcd4c06d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:34.331198 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb826c2-ebab-4344-addb-18b2dcd4c06d service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:34.341432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1262/2547] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:18:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:18:34.660056 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fa75c-da7e-4096-a36c-a416a024a0d4 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:34.661779 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-699fa75c-da7e-4096-a36c-a416a024a0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:34.676695 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fa75c-da7e-4096-a36c-a416a024a0d4 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:34.682636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169aa609-6c8d-4891-9755-c5e8250555dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:34.683223 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169aa609-6c8d-4891-9755-c5e8250555dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:34.683527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169aa609-6c8d-4891-9755-c5e8250555dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:34.683798 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169aa609-6c8d-4891-9755-c5e8250555dc admin admin] PATH_INFO: `/v3/projects/d04597cbe7ce4eb6b187949526c924ce` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:34.685020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169aa609-6c8d-4891-9755-c5e8250555dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d04597cbe7ce4eb6b187949526c924ce)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:34.687387 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169aa609-6c8d-4891-9755-c5e8250555dc admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:34.735864 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-169aa609-6c8d-4891-9755-c5e8250555dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd04597cbe7ce4eb6b187949526c924ce', 'request_id': 'req-169aa609-6c8d-4891-9755-c5e8250555dc'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:34.741249 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-169aa609-6c8d-4891-9755-c5e8250555dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd04597cbe7ce4eb6b187949526c924ce', 'request_id': 'req-169aa609-6c8d-4891-9755-c5e8250555dc'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:34.749999 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-169aa609-6c8d-4891-9755-c5e8250555dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd04597cbe7ce4eb6b187949526c924ce', 'request_id': 'req-169aa609-6c8d-4891-9755-c5e8250555dc'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:34.764372 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-169aa609-6c8d-4891-9755-c5e8250555dc admin admin] The token cache is being invalidate because project d04597cbe7ce4eb6b187949526c924ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:18:34.764733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-169aa609-6c8d-4891-9755-c5e8250555dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:34.765655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-169aa609-6c8d-4891-9755-c5e8250555dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:34.767616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1286/2548] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:18:34 2022] DELETE /identity/v3/projects/d04597cbe7ce4eb6b187949526c924ce => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:34.771817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb826c2-ebab-4344-addb-18b2dcd4c06d service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:34.783254 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb826c2-ebab-4344-addb-18b2dcd4c06d service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:34.844231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6e613-94bc-4385-94c2-3716cc2ac4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:34.845033 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6e613-94bc-4385-94c2-3716cc2ac4e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:34.845314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6e613-94bc-4385-94c2-3716cc2ac4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:34.845590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6e613-94bc-4385-94c2-3716cc2ac4e9 admin admin] PATH_INFO: `/v3/users/ec55bf68cafd4ccbae758c899a685ec6` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:34.846737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb6e613-94bc-4385-94c2-3716cc2ac4e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec55bf68cafd4ccbae758c899a685ec6)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:34.848737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb6e613-94bc-4385-94c2-3716cc2ac4e9 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:34.898944 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-2eb6e613-94bc-4385-94c2-3716cc2ac4e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec55bf68cafd4ccbae758c899a685ec6'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:34.899632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-2eb6e613-94bc-4385-94c2-3716cc2ac4e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec55bf68cafd4ccbae758c899a685ec6'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:34.900260 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-2eb6e613-94bc-4385-94c2-3716cc2ac4e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec55bf68cafd4ccbae758c899a685ec6'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:34.907214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-2eb6e613-94bc-4385-94c2-3716cc2ac4e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec55bf68cafd4ccbae758c899a685ec6'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:34.918767 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-2eb6e613-94bc-4385-94c2-3716cc2ac4e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec55bf68cafd4ccbae758c899a685ec6'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:34.933298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1263/2549] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:18:34 2022] DELETE /identity/v3/users/ec55bf68cafd4ccbae758c899a685ec6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:35.218648 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169aa609-6c8d-4891-9755-c5e8250555dc admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:35.220185 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-169aa609-6c8d-4891-9755-c5e8250555dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:35.233264 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169aa609-6c8d-4891-9755-c5e8250555dc admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:35.294242 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6ae21-2589-49f3-bdcc-e78f9d6134b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:35.294824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6ae21-2589-49f3-bdcc-e78f9d6134b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:35.295100 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6ae21-2589-49f3-bdcc-e78f9d6134b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:35.295371 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6ae21-2589-49f3-bdcc-e78f9d6134b7 admin admin] PATH_INFO: `/v3/projects/fcf026244e374b9198f1f75e6ee2ed99` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:35.296580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6ae21-2589-49f3-bdcc-e78f9d6134b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcf026244e374b9198f1f75e6ee2ed99)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:35.298574 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6ae21-2589-49f3-bdcc-e78f9d6134b7 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:35.349014 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5ac6ae21-2589-49f3-bdcc-e78f9d6134b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcf026244e374b9198f1f75e6ee2ed99', 'request_id': 'req-5ac6ae21-2589-49f3-bdcc-e78f9d6134b7'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:35.352954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:35.355207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5ac6ae21-2589-49f3-bdcc-e78f9d6134b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcf026244e374b9198f1f75e6ee2ed99', 'request_id': 'req-5ac6ae21-2589-49f3-bdcc-e78f9d6134b7'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:35.365674 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5ac6ae21-2589-49f3-bdcc-e78f9d6134b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcf026244e374b9198f1f75e6ee2ed99', 'request_id': 'req-5ac6ae21-2589-49f3-bdcc-e78f9d6134b7'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:35.377180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5ac6ae21-2589-49f3-bdcc-e78f9d6134b7 admin admin] The token cache is being invalidate because project fcf026244e374b9198f1f75e6ee2ed99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:18:35.377472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5ac6ae21-2589-49f3-bdcc-e78f9d6134b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:35.378191 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5ac6ae21-2589-49f3-bdcc-e78f9d6134b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:35.379827 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1287/2550] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:18:35 2022] DELETE /identity/v3/projects/fcf026244e374b9198f1f75e6ee2ed99 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:35.384998 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09a021-29d0-4e28-a92e-f914b4944f15 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:35.385491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09a021-29d0-4e28-a92e-f914b4944f15 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:35.385608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09a021-29d0-4e28-a92e-f914b4944f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:35.406655 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-cb09a021-29d0-4e28-a92e-f914b4944f15 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:35.407917 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-cb09a021-29d0-4e28-a92e-f914b4944f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:35.461434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:35.477371 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1264/2551] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:35.488532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e5a57-21f2-4847-95ff-c4b84aeb354d None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:35.488825 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e5a57-21f2-4847-95ff-c4b84aeb354d None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:35.489093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e5a57-21f2-4847-95ff-c4b84aeb354d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:35.509691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-075e5a57-21f2-4847-95ff-c4b84aeb354d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:35.510889 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-075e5a57-21f2-4847-95ff-c4b84aeb354d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:35.519367 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1288/2552] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:35.523807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09a021-29d0-4e28-a92e-f914b4944f15 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:35.534416 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09a021-29d0-4e28-a92e-f914b4944f15 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:35.540075 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1f9b7-3f39-45ae-a92b-a6eb5ba0ab77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:35.540678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1f9b7-3f39-45ae-a92b-a6eb5ba0ab77 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:35.540958 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1f9b7-3f39-45ae-a92b-a6eb5ba0ab77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:35.541231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1f9b7-3f39-45ae-a92b-a6eb5ba0ab77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:35.541881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1f9b7-3f39-45ae-a92b-a6eb5ba0ab77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:35.542183 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1f9b7-3f39-45ae-a92b-a6eb5ba0ab77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:35.543842 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1f9b7-3f39-45ae-a92b-a6eb5ba0ab77 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:35.557220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1265/2553] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:35.561517 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e5a57-21f2-4847-95ff-c4b84aeb354d None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:35.571162 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e5a57-21f2-4847-95ff-c4b84aeb354d None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:35.577129 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c217ffc-8402-47d4-a494-a3f65664eb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:35.577700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c217ffc-8402-47d4-a494-a3f65664eb99 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:35.577980 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c217ffc-8402-47d4-a494-a3f65664eb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:35.578256 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c217ffc-8402-47d4-a494-a3f65664eb99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:35.578871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c217ffc-8402-47d4-a494-a3f65664eb99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:35.580822 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c217ffc-8402-47d4-a494-a3f65664eb99 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:35.614563 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1289/2554] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:18:35 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:18:35.618638 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1f9b7-3f39-45ae-a92b-a6eb5ba0ab77 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:35.629511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1f9b7-3f39-45ae-a92b-a6eb5ba0ab77 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:35.692256 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77501f2e-1c15-49dc-b9e1-4f0340ab9ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:35.692986 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77501f2e-1c15-49dc-b9e1-4f0340ab9ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:35.693264 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77501f2e-1c15-49dc-b9e1-4f0340ab9ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:35.693535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77501f2e-1c15-49dc-b9e1-4f0340ab9ee6 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:35.694166 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77501f2e-1c15-49dc-b9e1-4f0340ab9ee6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:35.696188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77501f2e-1c15-49dc-b9e1-4f0340ab9ee6 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:35.721226 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1266/2555] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:18:35 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:35.725546 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c217ffc-8402-47d4-a494-a3f65664eb99 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:35.737582 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c217ffc-8402-47d4-a494-a3f65664eb99 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:35.742886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cda59c-0375-45e6-a8fc-49c2dd3250cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:35.743675 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cda59c-0375-45e6-a8fc-49c2dd3250cf admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:35.744006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cda59c-0375-45e6-a8fc-49c2dd3250cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:35.744306 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cda59c-0375-45e6-a8fc-49c2dd3250cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:35.744951 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cda59c-0375-45e6-a8fc-49c2dd3250cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:35.746628 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cda59c-0375-45e6-a8fc-49c2dd3250cf admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:35.758681 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1290/2556] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:18:35.762721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77501f2e-1c15-49dc-b9e1-4f0340ab9ee6 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:35.773215 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77501f2e-1c15-49dc-b9e1-4f0340ab9ee6 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:35.778079 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9529fe5-0717-4df0-b3f6-8cf53cd01199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:35.778676 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9529fe5-0717-4df0-b3f6-8cf53cd01199 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:35.778949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9529fe5-0717-4df0-b3f6-8cf53cd01199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:35.779217 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9529fe5-0717-4df0-b3f6-8cf53cd01199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:35.779896 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9529fe5-0717-4df0-b3f6-8cf53cd01199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:35.781605 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9529fe5-0717-4df0-b3f6-8cf53cd01199 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:35.792824 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1267/2557] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:35.796965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cda59c-0375-45e6-a8fc-49c2dd3250cf admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:35.807736 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cda59c-0375-45e6-a8fc-49c2dd3250cf admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:35.812248 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c733c96-a1be-453e-8b20-454b3950f9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:35.812976 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c733c96-a1be-453e-8b20-454b3950f9fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:35.813251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c733c96-a1be-453e-8b20-454b3950f9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:35.813618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c733c96-a1be-453e-8b20-454b3950f9fc admin admin] PATH_INFO: `/v3/projects/5b58a26ed97b40f38073be14bed8ab79/users/9c8ac5e4d3874f8a8520af0f04433ac1/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:35.828163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c733c96-a1be-453e-8b20-454b3950f9fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b58a26ed97b40f38073be14bed8ab79, user_id=9c8ac5e4d3874f8a8520af0f04433ac1, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:35.830881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c733c96-a1be-453e-8b20-454b3950f9fc admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:35.844872 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1291/2558] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:35 2022] PUT /identity/v3/projects/5b58a26ed97b40f38073be14bed8ab79/users/9c8ac5e4d3874f8a8520af0f04433ac1/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:35.855192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31ba1f-824e-4961-a678-f7da634a118b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:35.855512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31ba1f-824e-4961-a678-f7da634a118b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:35.855792 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31ba1f-824e-4961-a678-f7da634a118b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:35.889090 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-4a31ba1f-824e-4961-a678-f7da634a118b None None] MFA Rules not processed for user `9c8ac5e4d3874f8a8520af0f04433ac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:35.891135 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-4a31ba1f-824e-4961-a678-f7da634a118b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:35.966541 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1268/2559] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:35.971647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c733c96-a1be-453e-8b20-454b3950f9fc admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:36.038388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c733c96-a1be-453e-8b20-454b3950f9fc admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:36.043538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb993f-8910-486d-b26f-a49360d956ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:36.044136 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb993f-8910-486d-b26f-a49360d956ea admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:36.044406 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb993f-8910-486d-b26f-a49360d956ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:36.044710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb993f-8910-486d-b26f-a49360d956ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:36.045297 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb993f-8910-486d-b26f-a49360d956ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:36.047714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb993f-8910-486d-b26f-a49360d956ea admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:36.077800 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1292/2560] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:18:35 2022] POST /identity/v3/projects => generated 369 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:36.082302 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31ba1f-824e-4961-a678-f7da634a118b None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:36.093768 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31ba1f-824e-4961-a678-f7da634a118b None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:36.143357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:36.157361 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20481b3-b238-470a-b267-522cbcb5bc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:36.158002 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20481b3-b238-470a-b267-522cbcb5bc33 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:36.158280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20481b3-b238-470a-b267-522cbcb5bc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:36.158553 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20481b3-b238-470a-b267-522cbcb5bc33 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:36.159186 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20481b3-b238-470a-b267-522cbcb5bc33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:36.161168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20481b3-b238-470a-b267-522cbcb5bc33 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:36.186835 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1269/2561] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:18:36 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:36.192118 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb993f-8910-486d-b26f-a49360d956ea admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:36.206875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb993f-8910-486d-b26f-a49360d956ea admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:36.212864 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44cc1d7-f344-4d7e-a106-e5e085229325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:36.213463 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44cc1d7-f344-4d7e-a106-e5e085229325 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:36.213740 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44cc1d7-f344-4d7e-a106-e5e085229325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:36.214005 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44cc1d7-f344-4d7e-a106-e5e085229325 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:36.214632 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44cc1d7-f344-4d7e-a106-e5e085229325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:36.216258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44cc1d7-f344-4d7e-a106-e5e085229325 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:36.227399 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1293/2562] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:36.232791 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20481b3-b238-470a-b267-522cbcb5bc33 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:36.244521 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20481b3-b238-470a-b267-522cbcb5bc33 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:36.249490 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c5853-335f-49f2-959f-eafbd8c79b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:36.250146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c5853-335f-49f2-959f-eafbd8c79b8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:36.250669 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c5853-335f-49f2-959f-eafbd8c79b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:36.250973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c5853-335f-49f2-959f-eafbd8c79b8b admin admin] PATH_INFO: `/v3/projects/de17b1afab6248deb0f657bd4e143459/users/cb86d9e5446f49a18ad11e59b2cd6a20/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:36.266479 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c5853-335f-49f2-959f-eafbd8c79b8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=de17b1afab6248deb0f657bd4e143459, user_id=cb86d9e5446f49a18ad11e59b2cd6a20, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:36.269227 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c5853-335f-49f2-959f-eafbd8c79b8b admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:36.285007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1270/2563] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:36 2022] PUT /identity/v3/projects/de17b1afab6248deb0f657bd4e143459/users/cb86d9e5446f49a18ad11e59b2cd6a20/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:36.289246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44cc1d7-f344-4d7e-a106-e5e085229325 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:36.299691 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44cc1d7-f344-4d7e-a106-e5e085229325 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:36.351389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:36.362803 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed36ac-425c-431c-876c-b8aff8fbce83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:36.363465 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed36ac-425c-431c-876c-b8aff8fbce83 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:36.363751 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed36ac-425c-431c-876c-b8aff8fbce83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:36.364053 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed36ac-425c-431c-876c-b8aff8fbce83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:36.364699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed36ac-425c-431c-876c-b8aff8fbce83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:36.366323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed36ac-425c-431c-876c-b8aff8fbce83 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:36.377212 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1294/2564] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:36.381595 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c5853-335f-49f2-959f-eafbd8c79b8b admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:36.392653 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c5853-335f-49f2-959f-eafbd8c79b8b admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:36.397571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d578d-4e16-4d84-8983-7da83516814b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:36.398220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d578d-4e16-4d84-8983-7da83516814b admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:36.398498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d578d-4e16-4d84-8983-7da83516814b admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:36.398771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d578d-4e16-4d84-8983-7da83516814b admin admin] PATH_INFO: `/v3/projects/de17b1afab6248deb0f657bd4e143459/users/cb86d9e5446f49a18ad11e59b2cd6a20/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:36.400815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d578d-4e16-4d84-8983-7da83516814b admin admin] RBAC: Authorizing `identity:create_grant(project_id=de17b1afab6248deb0f657bd4e143459, user_id=cb86d9e5446f49a18ad11e59b2cd6a20, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:36.403524 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d578d-4e16-4d84-8983-7da83516814b admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:36.414488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1271/2565] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:36 2022] PUT /identity/v3/projects/de17b1afab6248deb0f657bd4e143459/users/cb86d9e5446f49a18ad11e59b2cd6a20/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:18:36.426229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3c01e-68a4-4985-8b91-885efd4b2bea None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:36.426517 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3c01e-68a4-4985-8b91-885efd4b2bea None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:36.426785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3c01e-68a4-4985-8b91-885efd4b2bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:36.456628 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-aca3c01e-68a4-4985-8b91-885efd4b2bea None None] MFA Rules not processed for user `cb86d9e5446f49a18ad11e59b2cd6a20`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:36.457894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-aca3c01e-68a4-4985-8b91-885efd4b2bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:36.532503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1295/2566] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:36.544107 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d578d-4e16-4d84-8983-7da83516814b admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:36.546343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-159d578d-4e16-4d84-8983-7da83516814b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:36.611026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d578d-4e16-4d84-8983-7da83516814b admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:36.615121 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1724b8-9405-448c-9a85-978e613a038b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:36.615809 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1724b8-9405-448c-9a85-978e613a038b service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:36.616068 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1724b8-9405-448c-9a85-978e613a038b service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:36.616341 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1724b8-9405-448c-9a85-978e613a038b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:36.626004 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1724b8-9405-448c-9a85-978e613a038b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:36.627460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1724b8-9405-448c-9a85-978e613a038b service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:36.637130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1272/2567] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:18:36 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:18:38.079969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3c01e-68a4-4985-8b91-885efd4b2bea None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:38.148616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3c01e-68a4-4985-8b91-885efd4b2bea None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:38.154087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3a58f-8bf4-48fa-bd22-6df490083193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:38.154729 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3a58f-8bf4-48fa-bd22-6df490083193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:38.155003 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3a58f-8bf4-48fa-bd22-6df490083193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:38.155274 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3a58f-8bf4-48fa-bd22-6df490083193 admin admin] PATH_INFO: `/v3/users/9c8ac5e4d3874f8a8520af0f04433ac1` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:38.156461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3a58f-8bf4-48fa-bd22-6df490083193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c8ac5e4d3874f8a8520af0f04433ac1)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:38.158423 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3a58f-8bf4-48fa-bd22-6df490083193 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:38.210745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-acd3a58f-8bf4-48fa-bd22-6df490083193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c8ac5e4d3874f8a8520af0f04433ac1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:38.211348 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-acd3a58f-8bf4-48fa-bd22-6df490083193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c8ac5e4d3874f8a8520af0f04433ac1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:38.211984 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-acd3a58f-8bf4-48fa-bd22-6df490083193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c8ac5e4d3874f8a8520af0f04433ac1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:38.219142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-acd3a58f-8bf4-48fa-bd22-6df490083193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c8ac5e4d3874f8a8520af0f04433ac1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:38.232731 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-acd3a58f-8bf4-48fa-bd22-6df490083193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c8ac5e4d3874f8a8520af0f04433ac1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:38.235540 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:38.247255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1296/2568] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:18:38 2022] DELETE /identity/v3/users/9c8ac5e4d3874f8a8520af0f04433ac1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:38.252030 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df66bd-67e1-4930-a7a2-014f532de732 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:38.252502 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df66bd-67e1-4930-a7a2-014f532de732 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:38.252642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df66bd-67e1-4930-a7a2-014f532de732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:38.271707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-82df66bd-67e1-4930-a7a2-014f532de732 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:38.272912 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-82df66bd-67e1-4930-a7a2-014f532de732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:38.333838 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1273/2569] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:18:38.345911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3a58f-8bf4-48fa-bd22-6df490083193 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:38.347973 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-acd3a58f-8bf4-48fa-bd22-6df490083193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:38.406586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3a58f-8bf4-48fa-bd22-6df490083193 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:38.410970 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de236b-81ce-4b26-8e1d-6402926531c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:38.411625 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de236b-81ce-4b26-8e1d-6402926531c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:38.411904 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de236b-81ce-4b26-8e1d-6402926531c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:38.412175 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de236b-81ce-4b26-8e1d-6402926531c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:38.422011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de236b-81ce-4b26-8e1d-6402926531c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:38.423480 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de236b-81ce-4b26-8e1d-6402926531c5 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:38.433578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1297/2570] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:18:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:18:38.734552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df66bd-67e1-4930-a7a2-014f532de732 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:38.736222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-82df66bd-67e1-4930-a7a2-014f532de732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:38.751393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df66bd-67e1-4930-a7a2-014f532de732 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:38.764093 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c363665-11b8-487a-a41a-70a61f0e7c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:38.764950 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c363665-11b8-487a-a41a-70a61f0e7c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:38.765413 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c363665-11b8-487a-a41a-70a61f0e7c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:38.765920 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c363665-11b8-487a-a41a-70a61f0e7c79 admin admin] PATH_INFO: `/v3/projects/5b58a26ed97b40f38073be14bed8ab79` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:38.767232 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c363665-11b8-487a-a41a-70a61f0e7c79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b58a26ed97b40f38073be14bed8ab79)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:38.769438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c363665-11b8-487a-a41a-70a61f0e7c79 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:38.812127 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:38.824289 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-9c363665-11b8-487a-a41a-70a61f0e7c79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b58a26ed97b40f38073be14bed8ab79', 'request_id': 'req-9c363665-11b8-487a-a41a-70a61f0e7c79'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:38.833683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-9c363665-11b8-487a-a41a-70a61f0e7c79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b58a26ed97b40f38073be14bed8ab79', 'request_id': 'req-9c363665-11b8-487a-a41a-70a61f0e7c79'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:38.842027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-9c363665-11b8-487a-a41a-70a61f0e7c79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b58a26ed97b40f38073be14bed8ab79', 'request_id': 'req-9c363665-11b8-487a-a41a-70a61f0e7c79'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:38.854262 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-9c363665-11b8-487a-a41a-70a61f0e7c79 admin admin] The token cache is being invalidate because project 5b58a26ed97b40f38073be14bed8ab79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:18:38.854550 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-9c363665-11b8-487a-a41a-70a61f0e7c79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:38.855197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-9c363665-11b8-487a-a41a-70a61f0e7c79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:38.856981 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1274/2571] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:18:38 2022] DELETE /identity/v3/projects/5b58a26ed97b40f38073be14bed8ab79 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:38.863968 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de236b-81ce-4b26-8e1d-6402926531c5 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:38.873991 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de236b-81ce-4b26-8e1d-6402926531c5 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:38.935210 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f7476-2c62-4abb-bbb8-6890ef0b765e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:38.935871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f7476-2c62-4abb-bbb8-6890ef0b765e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:38.936147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f7476-2c62-4abb-bbb8-6890ef0b765e admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:38.936415 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f7476-2c62-4abb-bbb8-6890ef0b765e admin admin] PATH_INFO: `/v3/users/cb86d9e5446f49a18ad11e59b2cd6a20` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:38.937564 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f7476-2c62-4abb-bbb8-6890ef0b765e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb86d9e5446f49a18ad11e59b2cd6a20)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:38.939573 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f7476-2c62-4abb-bbb8-6890ef0b765e admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:38.992214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-934f7476-2c62-4abb-bbb8-6890ef0b765e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb86d9e5446f49a18ad11e59b2cd6a20'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:38.992960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-934f7476-2c62-4abb-bbb8-6890ef0b765e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb86d9e5446f49a18ad11e59b2cd6a20'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:38.993672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-934f7476-2c62-4abb-bbb8-6890ef0b765e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb86d9e5446f49a18ad11e59b2cd6a20'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:39.001412 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-934f7476-2c62-4abb-bbb8-6890ef0b765e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb86d9e5446f49a18ad11e59b2cd6a20'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:39.014663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-934f7476-2c62-4abb-bbb8-6890ef0b765e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb86d9e5446f49a18ad11e59b2cd6a20'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:39.033846 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1298/2572] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:18:38 2022] DELETE /identity/v3/users/cb86d9e5446f49a18ad11e59b2cd6a20 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:39.557939 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c363665-11b8-487a-a41a-70a61f0e7c79 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:39.559486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-9c363665-11b8-487a-a41a-70a61f0e7c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:39.573079 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c363665-11b8-487a-a41a-70a61f0e7c79 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:39.654586 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0065583-fe4b-4f21-ab76-73c81d154e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:39.655864 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0065583-fe4b-4f21-ab76-73c81d154e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:39.656299 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0065583-fe4b-4f21-ab76-73c81d154e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:39.656750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0065583-fe4b-4f21-ab76-73c81d154e30 admin admin] PATH_INFO: `/v3/projects/de17b1afab6248deb0f657bd4e143459` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:39.658123 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0065583-fe4b-4f21-ab76-73c81d154e30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de17b1afab6248deb0f657bd4e143459)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:39.660537 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0065583-fe4b-4f21-ab76-73c81d154e30 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:39.714511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e0065583-fe4b-4f21-ab76-73c81d154e30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de17b1afab6248deb0f657bd4e143459', 'request_id': 'req-e0065583-fe4b-4f21-ab76-73c81d154e30'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:39.720554 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e0065583-fe4b-4f21-ab76-73c81d154e30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de17b1afab6248deb0f657bd4e143459', 'request_id': 'req-e0065583-fe4b-4f21-ab76-73c81d154e30'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:39.732326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e0065583-fe4b-4f21-ab76-73c81d154e30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de17b1afab6248deb0f657bd4e143459', 'request_id': 'req-e0065583-fe4b-4f21-ab76-73c81d154e30'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:39.745696 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e0065583-fe4b-4f21-ab76-73c81d154e30 admin admin] The token cache is being invalidate because project de17b1afab6248deb0f657bd4e143459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:18:39.746003 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e0065583-fe4b-4f21-ab76-73c81d154e30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:39.746889 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e0065583-fe4b-4f21-ab76-73c81d154e30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:39.752147 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1275/2573] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:18:39 2022] DELETE /identity/v3/projects/de17b1afab6248deb0f657bd4e143459 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:39.753825 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8aca11-c293-4f80-9d8b-40884142a37c None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:39.754290 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8aca11-c293-4f80-9d8b-40884142a37c None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:39.754385 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8aca11-c293-4f80-9d8b-40884142a37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:39.786496 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-3a8aca11-c293-4f80-9d8b-40884142a37c None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:39.787811 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3a8aca11-c293-4f80-9d8b-40884142a37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:39.800284 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:39.888613 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1299/2574] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:39.890505 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e3edd-1f32-4ec0-97dc-e02eebd6846d None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:39.890505 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e3edd-1f32-4ec0-97dc-e02eebd6846d None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:39.890748 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e3edd-1f32-4ec0-97dc-e02eebd6846d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:39.938357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-2f0e3edd-1f32-4ec0-97dc-e02eebd6846d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:39.939771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2f0e3edd-1f32-4ec0-97dc-e02eebd6846d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:39.959845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1276/2575] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:39.964462 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8aca11-c293-4f80-9d8b-40884142a37c None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:39.983445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8aca11-c293-4f80-9d8b-40884142a37c None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:39.988576 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bb922-ed11-4e87-830d-aa7b9e720c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:39.989287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bb922-ed11-4e87-830d-aa7b9e720c99 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:39.989577 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bb922-ed11-4e87-830d-aa7b9e720c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:39.989852 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bb922-ed11-4e87-830d-aa7b9e720c99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:39.990530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bb922-ed11-4e87-830d-aa7b9e720c99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:39.990831 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bb922-ed11-4e87-830d-aa7b9e720c99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:39.992588 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bb922-ed11-4e87-830d-aa7b9e720c99 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:40.007572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1300/2576] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:40.011902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e3edd-1f32-4ec0-97dc-e02eebd6846d None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:40.023188 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e3edd-1f32-4ec0-97dc-e02eebd6846d None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:40.028601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7cb65-f971-490f-b76a-aee737d3c2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:40.029251 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7cb65-f971-490f-b76a-aee737d3c2f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:40.029529 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7cb65-f971-490f-b76a-aee737d3c2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:40.029797 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7cb65-f971-490f-b76a-aee737d3c2f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:40.030455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b7cb65-f971-490f-b76a-aee737d3c2f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:40.032589 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b7cb65-f971-490f-b76a-aee737d3c2f6 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:40.063658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1277/2577] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:18:40 2022] POST /identity/v3/projects => generated 393 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:18:40.068508 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bb922-ed11-4e87-830d-aa7b9e720c99 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:40.079674 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bb922-ed11-4e87-830d-aa7b9e720c99 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:40.152110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcb727-7f91-4859-8910-612f6b4a34f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:40.152792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcb727-7f91-4859-8910-612f6b4a34f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:40.153137 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcb727-7f91-4859-8910-612f6b4a34f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:40.153473 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcb727-7f91-4859-8910-612f6b4a34f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:40.154192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fcb727-7f91-4859-8910-612f6b4a34f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:40.156581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fcb727-7f91-4859-8910-612f6b4a34f1 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:40.187446 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1301/2578] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:18:40 2022] POST /identity/v3/users => generated 297 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:40.191324 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7cb65-f971-490f-b76a-aee737d3c2f6 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:40.216696 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7cb65-f971-490f-b76a-aee737d3c2f6 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:40.228697 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3107330-5869-40ce-b9f0-03fd6c5224ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:40.229335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3107330-5869-40ce-b9f0-03fd6c5224ef admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:40.229608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3107330-5869-40ce-b9f0-03fd6c5224ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:40.229879 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3107330-5869-40ce-b9f0-03fd6c5224ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:40.230535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3107330-5869-40ce-b9f0-03fd6c5224ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:40.232194 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3107330-5869-40ce-b9f0-03fd6c5224ef admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:40.244811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1278/2579] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:40.248624 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcb727-7f91-4859-8910-612f6b4a34f1 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:40.265654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcb727-7f91-4859-8910-612f6b4a34f1 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:40.271351 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c71d1-f4b9-4e65-9565-36003799f6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:40.272032 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c71d1-f4b9-4e65-9565-36003799f6d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:40.272309 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c71d1-f4b9-4e65-9565-36003799f6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:40.272579 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c71d1-f4b9-4e65-9565-36003799f6d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:40.273409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94c71d1-f4b9-4e65-9565-36003799f6d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:40.274846 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94c71d1-f4b9-4e65-9565-36003799f6d3 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:40.289066 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1302/2580] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:18:40.292976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3107330-5869-40ce-b9f0-03fd6c5224ef admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:40.304876 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3107330-5869-40ce-b9f0-03fd6c5224ef admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:40.310498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa87403-f62d-49ef-bcb0-93beea00ef0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:40.311166 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa87403-f62d-49ef-bcb0-93beea00ef0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:40.311488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa87403-f62d-49ef-bcb0-93beea00ef0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:40.312107 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa87403-f62d-49ef-bcb0-93beea00ef0c admin admin] PATH_INFO: `/v3/projects/d49f6850f25d4723a22dff2d39149088/users/6f7db263dac64230a1685269eb213cfa/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:40.328383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa87403-f62d-49ef-bcb0-93beea00ef0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d49f6850f25d4723a22dff2d39149088, user_id=6f7db263dac64230a1685269eb213cfa, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:40.331850 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa87403-f62d-49ef-bcb0-93beea00ef0c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:40.334337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:40.354123 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1279/2581] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:40 2022] PUT /identity/v3/projects/d49f6850f25d4723a22dff2d39149088/users/6f7db263dac64230a1685269eb213cfa/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:40.364904 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d43db-5c92-4625-9f78-89fa9a3f33cf None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:40.365232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d43db-5c92-4625-9f78-89fa9a3f33cf None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:40.365454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d43db-5c92-4625-9f78-89fa9a3f33cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:40.400198 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-615d43db-5c92-4625-9f78-89fa9a3f33cf None None] MFA Rules not processed for user `6f7db263dac64230a1685269eb213cfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:40.401640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-615d43db-5c92-4625-9f78-89fa9a3f33cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:40.526747 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1303/2582] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:40.531439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa87403-f62d-49ef-bcb0-93beea00ef0c admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:40.608517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa87403-f62d-49ef-bcb0-93beea00ef0c admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:40.614119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fa5e4-caab-4814-9ea5-8f8579a7c5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:40.615000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fa5e4-caab-4814-9ea5-8f8579a7c5ba admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:40.615352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fa5e4-caab-4814-9ea5-8f8579a7c5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:40.615731 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fa5e4-caab-4814-9ea5-8f8579a7c5ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:40.616451 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fa5e4-caab-4814-9ea5-8f8579a7c5ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:40.618458 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fa5e4-caab-4814-9ea5-8f8579a7c5ba admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:40.651720 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1280/2583] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:18:40 2022] POST /identity/v3/projects => generated 391 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:40.653148 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d43db-5c92-4625-9f78-89fa9a3f33cf None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:40.666143 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d43db-5c92-4625-9f78-89fa9a3f33cf None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:40.675331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:40.729149 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4bcf9-9a2f-4f08-8298-f0312a351a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:40.729761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4bcf9-9a2f-4f08-8298-f0312a351a46 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:40.730037 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4bcf9-9a2f-4f08-8298-f0312a351a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:40.730305 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4bcf9-9a2f-4f08-8298-f0312a351a46 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:40.730933 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e4bcf9-9a2f-4f08-8298-f0312a351a46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:40.732899 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e4bcf9-9a2f-4f08-8298-f0312a351a46 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:40.757120 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1304/2584] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:18:40 2022] POST /identity/v3/users => generated 302 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:40.762236 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fa5e4-caab-4814-9ea5-8f8579a7c5ba admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:40.774939 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fa5e4-caab-4814-9ea5-8f8579a7c5ba admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:40.781147 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04714971-d064-43e8-8d21-be20edcb965a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:40.781757 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04714971-d064-43e8-8d21-be20edcb965a admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:40.782039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04714971-d064-43e8-8d21-be20edcb965a admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:40.782311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04714971-d064-43e8-8d21-be20edcb965a admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:40.782959 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04714971-d064-43e8-8d21-be20edcb965a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:40.784917 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04714971-d064-43e8-8d21-be20edcb965a admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:40.796586 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1281/2585] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:40.801355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4bcf9-9a2f-4f08-8298-f0312a351a46 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:40.811938 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4bcf9-9a2f-4f08-8298-f0312a351a46 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:40.816689 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6cff06-a03f-48c0-be0d-c3dfea0bf4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:40.817361 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6cff06-a03f-48c0-be0d-c3dfea0bf4aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:40.817636 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6cff06-a03f-48c0-be0d-c3dfea0bf4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:40.817912 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6cff06-a03f-48c0-be0d-c3dfea0bf4aa admin admin] PATH_INFO: `/v3/projects/ead3d12587b24bf5b99ca9bc35d4c1c4/users/2482494ff458428ea7259486cabdbc41/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:40.833383 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6cff06-a03f-48c0-be0d-c3dfea0bf4aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ead3d12587b24bf5b99ca9bc35d4c1c4, user_id=2482494ff458428ea7259486cabdbc41, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:40.836144 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6cff06-a03f-48c0-be0d-c3dfea0bf4aa admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:40.850349 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1305/2586] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:40 2022] PUT /identity/v3/projects/ead3d12587b24bf5b99ca9bc35d4c1c4/users/2482494ff458428ea7259486cabdbc41/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:40.854750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04714971-d064-43e8-8d21-be20edcb965a admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:40.866221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04714971-d064-43e8-8d21-be20edcb965a admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:40.925234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030adba-e1f4-4dab-ac2f-459846cc1a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:40.925857 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030adba-e1f4-4dab-ac2f-459846cc1a0b admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:40.926139 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030adba-e1f4-4dab-ac2f-459846cc1a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:40.926412 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030adba-e1f4-4dab-ac2f-459846cc1a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:40.927068 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1030adba-e1f4-4dab-ac2f-459846cc1a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:40.928703 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1030adba-e1f4-4dab-ac2f-459846cc1a0b admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:40.939166 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1282/2587] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:40.943662 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6cff06-a03f-48c0-be0d-c3dfea0bf4aa admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:40.954455 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6cff06-a03f-48c0-be0d-c3dfea0bf4aa admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:40.960011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342f6ad-615a-4f8e-9fcf-53bd0335edc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:40.960666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342f6ad-615a-4f8e-9fcf-53bd0335edc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:40.960944 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342f6ad-615a-4f8e-9fcf-53bd0335edc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:40.961216 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342f6ad-615a-4f8e-9fcf-53bd0335edc6 admin admin] PATH_INFO: `/v3/projects/ead3d12587b24bf5b99ca9bc35d4c1c4/users/2482494ff458428ea7259486cabdbc41/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:40.963479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342f6ad-615a-4f8e-9fcf-53bd0335edc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ead3d12587b24bf5b99ca9bc35d4c1c4, user_id=2482494ff458428ea7259486cabdbc41, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:40.966112 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342f6ad-615a-4f8e-9fcf-53bd0335edc6 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:40.977093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1306/2588] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:40 2022] PUT /identity/v3/projects/ead3d12587b24bf5b99ca9bc35d4c1c4/users/2482494ff458428ea7259486cabdbc41/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:40.988706 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9cd49f-c4c3-46a3-9762-c335b15dcf99 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:40.989103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9cd49f-c4c3-46a3-9762-c335b15dcf99 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:40.989379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9cd49f-c4c3-46a3-9762-c335b15dcf99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:41.019580 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-2c9cd49f-c4c3-46a3-9762-c335b15dcf99 None None] MFA Rules not processed for user `2482494ff458428ea7259486cabdbc41`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:41.020809 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2c9cd49f-c4c3-46a3-9762-c335b15dcf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:41.080057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:41.106458 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1283/2589] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:40 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:41.117633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342f6ad-615a-4f8e-9fcf-53bd0335edc6 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:41.119616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b342f6ad-615a-4f8e-9fcf-53bd0335edc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:41.201068 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342f6ad-615a-4f8e-9fcf-53bd0335edc6 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:41.204862 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7807d-a726-4919-9f09-1be343dbb839 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:41.205557 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7807d-a726-4919-9f09-1be343dbb839 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:41.205837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7807d-a726-4919-9f09-1be343dbb839 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:41.206118 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7807d-a726-4919-9f09-1be343dbb839 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:41.216494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff7807d-a726-4919-9f09-1be343dbb839 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:41.218009 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff7807d-a726-4919-9f09-1be343dbb839 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:41.228427 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1307/2590] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:18:41 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:18:41.281852 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9cd49f-c4c3-46a3-9762-c335b15dcf99 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:41.283601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2c9cd49f-c4c3-46a3-9762-c335b15dcf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:41.296639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9cd49f-c4c3-46a3-9762-c335b15dcf99 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:41.300703 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf892272-deb1-4ca9-95ab-945c006efd3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:41.301403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf892272-deb1-4ca9-95ab-945c006efd3f service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:41.301682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf892272-deb1-4ca9-95ab-945c006efd3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:41.301957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf892272-deb1-4ca9-95ab-945c006efd3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:41.367520 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf892272-deb1-4ca9-95ab-945c006efd3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:41.368996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf892272-deb1-4ca9-95ab-945c006efd3f service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:41.378757 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1284/2591] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:18:41 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:18:43.184618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb843bc-8ccf-4cb8-b57c-621de2f052ca None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:43.185180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb843bc-8ccf-4cb8-b57c-621de2f052ca None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:43.185624 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb843bc-8ccf-4cb8-b57c-621de2f052ca None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:43.187657 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1308/2592] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 23 07:18:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:18:43.194078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a5c88-3c29-4e70-be45-597155f1f426 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:43.194589 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a5c88-3c29-4e70-be45-597155f1f426 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:43.194682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a5c88-3c29-4e70-be45-597155f1f426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:43.216212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-c52a5c88-3c29-4e70-be45-597155f1f426 None None] MFA Rules not processed for user `938364c02d8d4dd2ae832c5129be3cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:43.217460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c52a5c88-3c29-4e70-be45-597155f1f426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:43.285504 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1285/2593] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 23 07:18:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:18:43.302127 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb843bc-8ccf-4cb8-b57c-621de2f052ca None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:43.303929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-deb843bc-8ccf-4cb8-b57c-621de2f052ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:43.326549 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb843bc-8ccf-4cb8-b57c-621de2f052ca None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:43.331810 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c57548-ceb7-491d-97d3-8c2e2353824d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:43.333037 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c57548-ceb7-491d-97d3-8c2e2353824d service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:43.333571 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c57548-ceb7-491d-97d3-8c2e2353824d service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:43.333923 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c57548-ceb7-491d-97d3-8c2e2353824d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:43.357697 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c57548-ceb7-491d-97d3-8c2e2353824d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:43.359604 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c57548-ceb7-491d-97d3-8c2e2353824d service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:43.375679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1309/2594] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:18:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:18:43.402433 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36766927-5e49-451a-aed4-fe7241cc23e1 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:43.402832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36766927-5e49-451a-aed4-fe7241cc23e1 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:43.403160 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36766927-5e49-451a-aed4-fe7241cc23e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:43.422733 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-36766927-5e49-451a-aed4-fe7241cc23e1 None None] MFA Rules not processed for user `ffd12f09606b4e6ba12f63b976df1884`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:43.424047 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-36766927-5e49-451a-aed4-fe7241cc23e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:43.469303 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfd213-7c35-49b1-8ee3-621dad9e7e22 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:43.469591 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfd213-7c35-49b1-8ee3-621dad9e7e22 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:43.469872 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfd213-7c35-49b1-8ee3-621dad9e7e22 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:43.471530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1310/2595] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 23 07:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:18:43.476924 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b3b35-1d9b-4d37-bbb5-10d3bdac1ed9 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:43.477205 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b3b35-1d9b-4d37-bbb5-10d3bdac1ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:43.477479 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b3b35-1d9b-4d37-bbb5-10d3bdac1ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:43.497359 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-6b8b3b35-1d9b-4d37-bbb5-10d3bdac1ed9 None None] MFA Rules not processed for user `a8d2cd93042c4583b63afe629d47fce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:43.498576 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6b8b3b35-1d9b-4d37-bbb5-10d3bdac1ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:43.513017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1286/2596] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:43.561087 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:43.587751 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1311/2597] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 23 07:18:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:18:43.599547 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36766927-5e49-451a-aed4-fe7241cc23e1 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:43.619468 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36766927-5e49-451a-aed4-fe7241cc23e1 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:43.624751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25372a8-b1c5-4c41-9da1-4bb4540b14cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:43.625349 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25372a8-b1c5-4c41-9da1-4bb4540b14cd service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:43.625624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25372a8-b1c5-4c41-9da1-4bb4540b14cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:43.625892 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25372a8-b1c5-4c41-9da1-4bb4540b14cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:43.637684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25372a8-b1c5-4c41-9da1-4bb4540b14cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:43.639210 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25372a8-b1c5-4c41-9da1-4bb4540b14cd service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:43.655312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1287/2598] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:18:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:18:44.882574 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b3b35-1d9b-4d37-bbb5-10d3bdac1ed9 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:44.949320 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b3b35-1d9b-4d37-bbb5-10d3bdac1ed9 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:44.954130 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006aa3e-351a-43f0-adc8-999a534ea7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:44.955854 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006aa3e-351a-43f0-adc8-999a534ea7ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:44.956070 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006aa3e-351a-43f0-adc8-999a534ea7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:44.956348 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006aa3e-351a-43f0-adc8-999a534ea7ef admin admin] PATH_INFO: `/v3/users/6f7db263dac64230a1685269eb213cfa` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:44.957506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006aa3e-351a-43f0-adc8-999a534ea7ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f7db263dac64230a1685269eb213cfa)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:44.959528 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006aa3e-351a-43f0-adc8-999a534ea7ef admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:45.008945 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1006aa3e-351a-43f0-adc8-999a534ea7ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f7db263dac64230a1685269eb213cfa'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:45.009594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1006aa3e-351a-43f0-adc8-999a534ea7ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f7db263dac64230a1685269eb213cfa'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:45.010207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1006aa3e-351a-43f0-adc8-999a534ea7ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f7db263dac64230a1685269eb213cfa'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:45.018884 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1006aa3e-351a-43f0-adc8-999a534ea7ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f7db263dac64230a1685269eb213cfa'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:45.031129 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1006aa3e-351a-43f0-adc8-999a534ea7ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f7db263dac64230a1685269eb213cfa'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:45.046211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1312/2599] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:18:44 2022] DELETE /identity/v3/users/6f7db263dac64230a1685269eb213cfa => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:45.050741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55363b62-8d1a-4f58-8d1e-ef01b134c756 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:45.051255 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55363b62-8d1a-4f58-8d1e-ef01b134c756 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:45.051379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55363b62-8d1a-4f58-8d1e-ef01b134c756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:45.072213 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-55363b62-8d1a-4f58-8d1e-ef01b134c756 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:45.073457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-55363b62-8d1a-4f58-8d1e-ef01b134c756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:45.102846 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:45.140175 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1288/2600] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:45.166122 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006aa3e-351a-43f0-adc8-999a534ea7ef admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:45.168509 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-1006aa3e-351a-43f0-adc8-999a534ea7ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:45.230875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006aa3e-351a-43f0-adc8-999a534ea7ef admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:45.234847 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae184e-9cab-42a5-abd5-acc575fd5b28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:45.235549 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae184e-9cab-42a5-abd5-acc575fd5b28 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:45.235893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae184e-9cab-42a5-abd5-acc575fd5b28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:45.236235 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae184e-9cab-42a5-abd5-acc575fd5b28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:45.246895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae184e-9cab-42a5-abd5-acc575fd5b28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:45.248784 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae184e-9cab-42a5-abd5-acc575fd5b28 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:45.259474 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1313/2601] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:18:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:18:45.664063 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55363b62-8d1a-4f58-8d1e-ef01b134c756 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:45.665692 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-55363b62-8d1a-4f58-8d1e-ef01b134c756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:45.689616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55363b62-8d1a-4f58-8d1e-ef01b134c756 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:45.702146 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56baeb3-a2ae-4836-854f-4d3c986917c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:45.702904 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56baeb3-a2ae-4836-854f-4d3c986917c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:45.703247 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56baeb3-a2ae-4836-854f-4d3c986917c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:45.703623 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56baeb3-a2ae-4836-854f-4d3c986917c9 admin admin] PATH_INFO: `/v3/projects/d49f6850f25d4723a22dff2d39149088` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:45.704892 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56baeb3-a2ae-4836-854f-4d3c986917c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d49f6850f25d4723a22dff2d39149088)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:45.708113 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56baeb3-a2ae-4836-854f-4d3c986917c9 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:45.783981 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-c56baeb3-a2ae-4836-854f-4d3c986917c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd49f6850f25d4723a22dff2d39149088', 'request_id': 'req-c56baeb3-a2ae-4836-854f-4d3c986917c9'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:45.792705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-c56baeb3-a2ae-4836-854f-4d3c986917c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd49f6850f25d4723a22dff2d39149088', 'request_id': 'req-c56baeb3-a2ae-4836-854f-4d3c986917c9'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:45.803759 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-c56baeb3-a2ae-4836-854f-4d3c986917c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd49f6850f25d4723a22dff2d39149088', 'request_id': 'req-c56baeb3-a2ae-4836-854f-4d3c986917c9'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:45.824741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-c56baeb3-a2ae-4836-854f-4d3c986917c9 admin admin] The token cache is being invalidate because project d49f6850f25d4723a22dff2d39149088 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:18:45.825318 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-c56baeb3-a2ae-4836-854f-4d3c986917c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:45.826671 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-c56baeb3-a2ae-4836-854f-4d3c986917c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:45.829049 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1289/2602] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:18:45 2022] DELETE /identity/v3/projects/d49f6850f25d4723a22dff2d39149088 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:45.833778 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae184e-9cab-42a5-abd5-acc575fd5b28 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:45.863932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae184e-9cab-42a5-abd5-acc575fd5b28 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:45.868165 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:45.943831 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058495cc-5c25-4f2e-bec3-ce70f0c6b317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:45.944486 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058495cc-5c25-4f2e-bec3-ce70f0c6b317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:45.944909 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058495cc-5c25-4f2e-bec3-ce70f0c6b317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:45.945327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058495cc-5c25-4f2e-bec3-ce70f0c6b317 admin admin] PATH_INFO: `/v3/users/2482494ff458428ea7259486cabdbc41` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:45.946497 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058495cc-5c25-4f2e-bec3-ce70f0c6b317 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2482494ff458428ea7259486cabdbc41)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:45.948685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058495cc-5c25-4f2e-bec3-ce70f0c6b317 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:46.006161 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-058495cc-5c25-4f2e-bec3-ce70f0c6b317 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2482494ff458428ea7259486cabdbc41'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:46.006858 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-058495cc-5c25-4f2e-bec3-ce70f0c6b317 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2482494ff458428ea7259486cabdbc41'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:46.007646 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-058495cc-5c25-4f2e-bec3-ce70f0c6b317 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2482494ff458428ea7259486cabdbc41'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:46.016343 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-058495cc-5c25-4f2e-bec3-ce70f0c6b317 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2482494ff458428ea7259486cabdbc41'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:46.032645 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-058495cc-5c25-4f2e-bec3-ce70f0c6b317 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2482494ff458428ea7259486cabdbc41'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:46.048709 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1314/2603] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:18:45 2022] DELETE /identity/v3/users/2482494ff458428ea7259486cabdbc41 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:46.526367 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56baeb3-a2ae-4836-854f-4d3c986917c9 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:46.527872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-c56baeb3-a2ae-4836-854f-4d3c986917c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:46.542109 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56baeb3-a2ae-4836-854f-4d3c986917c9 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:46.609344 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4142c3a-6ba0-4f8e-ba99-81051d1716e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:46.609994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4142c3a-6ba0-4f8e-ba99-81051d1716e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:46.610257 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4142c3a-6ba0-4f8e-ba99-81051d1716e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:46.610547 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4142c3a-6ba0-4f8e-ba99-81051d1716e6 admin admin] PATH_INFO: `/v3/projects/ead3d12587b24bf5b99ca9bc35d4c1c4` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:46.611893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4142c3a-6ba0-4f8e-ba99-81051d1716e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ead3d12587b24bf5b99ca9bc35d4c1c4)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:46.613922 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4142c3a-6ba0-4f8e-ba99-81051d1716e6 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:46.679334 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d4142c3a-6ba0-4f8e-ba99-81051d1716e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ead3d12587b24bf5b99ca9bc35d4c1c4', 'request_id': 'req-d4142c3a-6ba0-4f8e-ba99-81051d1716e6'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:46.691168 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:46.695264 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d4142c3a-6ba0-4f8e-ba99-81051d1716e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ead3d12587b24bf5b99ca9bc35d4c1c4', 'request_id': 'req-d4142c3a-6ba0-4f8e-ba99-81051d1716e6'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:46.706298 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d4142c3a-6ba0-4f8e-ba99-81051d1716e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ead3d12587b24bf5b99ca9bc35d4c1c4', 'request_id': 'req-d4142c3a-6ba0-4f8e-ba99-81051d1716e6'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:46.727308 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d4142c3a-6ba0-4f8e-ba99-81051d1716e6 admin admin] The token cache is being invalidate because project ead3d12587b24bf5b99ca9bc35d4c1c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:18:46.727758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d4142c3a-6ba0-4f8e-ba99-81051d1716e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:46.728717 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d4142c3a-6ba0-4f8e-ba99-81051d1716e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:46.731206 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1290/2604] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:18:46 2022] DELETE /identity/v3/projects/ead3d12587b24bf5b99ca9bc35d4c1c4 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:46.736570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a322e-ea8d-4692-b170-6e9466c7000a None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:46.736856 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a322e-ea8d-4692-b170-6e9466c7000a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:46.737127 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a322e-ea8d-4692-b170-6e9466c7000a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:46.758508 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-fa9a322e-ea8d-4692-b170-6e9466c7000a None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:46.760094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-fa9a322e-ea8d-4692-b170-6e9466c7000a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:46.847494 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1315/2605] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:46.849008 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed71333-3e8e-47d6-b9cd-747baf952abd None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:46.849415 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed71333-3e8e-47d6-b9cd-747baf952abd None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:46.849674 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed71333-3e8e-47d6-b9cd-747baf952abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:46.874712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-0ed71333-3e8e-47d6-b9cd-747baf952abd None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:46.876273 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-0ed71333-3e8e-47d6-b9cd-747baf952abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:46.889434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1291/2606] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:46.894047 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a322e-ea8d-4692-b170-6e9466c7000a None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:46.904334 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a322e-ea8d-4692-b170-6e9466c7000a None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:46.912034 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec66781-2440-46ad-96e9-25244de748f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:46.912993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec66781-2440-46ad-96e9-25244de748f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:46.913720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec66781-2440-46ad-96e9-25244de748f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:46.914219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec66781-2440-46ad-96e9-25244de748f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:46.915114 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec66781-2440-46ad-96e9-25244de748f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:46.915689 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec66781-2440-46ad-96e9-25244de748f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:46.924193 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec66781-2440-46ad-96e9-25244de748f8 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:46.952655 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1316/2607] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:18:46.954778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed71333-3e8e-47d6-b9cd-747baf952abd None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:46.966395 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed71333-3e8e-47d6-b9cd-747baf952abd None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:46.974072 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dc1d9-7a24-4877-bf82-6d497b4f261c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:46.974755 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dc1d9-7a24-4877-bf82-6d497b4f261c admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:46.975517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dc1d9-7a24-4877-bf82-6d497b4f261c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:46.975794 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dc1d9-7a24-4877-bf82-6d497b4f261c admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:46.976439 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dc1d9-7a24-4877-bf82-6d497b4f261c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:46.978775 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dc1d9-7a24-4877-bf82-6d497b4f261c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:47.023336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1292/2608] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:18:46 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:18:47.024539 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec66781-2440-46ad-96e9-25244de748f8 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:47.035191 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec66781-2440-46ad-96e9-25244de748f8 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:47.044573 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:47.103134 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd549bf-a585-4095-8433-608894d194ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:47.103678 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd549bf-a585-4095-8433-608894d194ea admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:47.103945 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd549bf-a585-4095-8433-608894d194ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:47.104252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd549bf-a585-4095-8433-608894d194ea admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:47.105015 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd549bf-a585-4095-8433-608894d194ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:47.107226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd549bf-a585-4095-8433-608894d194ea admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:47.132208 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1317/2609] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:18:47 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:47.136529 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dc1d9-7a24-4877-bf82-6d497b4f261c admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:47.147827 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dc1d9-7a24-4877-bf82-6d497b4f261c admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:47.153283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e98378-8bc4-47f3-bffe-300ca9c21cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:47.153894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e98378-8bc4-47f3-bffe-300ca9c21cfc admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:47.154172 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e98378-8bc4-47f3-bffe-300ca9c21cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:47.154443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e98378-8bc4-47f3-bffe-300ca9c21cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:47.155088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e98378-8bc4-47f3-bffe-300ca9c21cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:47.156816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e98378-8bc4-47f3-bffe-300ca9c21cfc admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:47.171201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1293/2610] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:47.172478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd549bf-a585-4095-8433-608894d194ea admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:47.183232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd549bf-a585-4095-8433-608894d194ea admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:47.190095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0877d8e-04c9-4072-b9d3-0a9c3aa9b537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:47.191117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0877d8e-04c9-4072-b9d3-0a9c3aa9b537 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:47.191439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0877d8e-04c9-4072-b9d3-0a9c3aa9b537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:47.191816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0877d8e-04c9-4072-b9d3-0a9c3aa9b537 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:47.192618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0877d8e-04c9-4072-b9d3-0a9c3aa9b537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:47.195044 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0877d8e-04c9-4072-b9d3-0a9c3aa9b537 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:47.207062 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1318/2611] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:47.211463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e98378-8bc4-47f3-bffe-300ca9c21cfc admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:47.222172 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e98378-8bc4-47f3-bffe-300ca9c21cfc admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:47.227079 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf2cb4-769f-42a4-bd82-3dac80ed345c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:47.227845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf2cb4-769f-42a4-bd82-3dac80ed345c admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:47.228114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf2cb4-769f-42a4-bd82-3dac80ed345c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:47.228386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf2cb4-769f-42a4-bd82-3dac80ed345c admin admin] PATH_INFO: `/v3/projects/a7c015ee4d634bc0842620a32509c961/users/de88d4b8b83043a19909f3c7c41b8287/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:47.245196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf2cb4-769f-42a4-bd82-3dac80ed345c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7c015ee4d634bc0842620a32509c961, user_id=de88d4b8b83043a19909f3c7c41b8287, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:47.247917 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf2cb4-769f-42a4-bd82-3dac80ed345c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:47.267687 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1294/2612] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:47 2022] PUT /identity/v3/projects/a7c015ee4d634bc0842620a32509c961/users/de88d4b8b83043a19909f3c7c41b8287/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:47.279210 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d4bf9-0f4a-49a4-bdda-85a456414cbf None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:47.279561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d4bf9-0f4a-49a4-bdda-85a456414cbf None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:47.279837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d4bf9-0f4a-49a4-bdda-85a456414cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:47.312435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-552d4bf9-0f4a-49a4-bdda-85a456414cbf None None] MFA Rules not processed for user `de88d4b8b83043a19909f3c7c41b8287`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:47.313656 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-552d4bf9-0f4a-49a4-bdda-85a456414cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:47.388034 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1319/2613] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:47.392563 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf2cb4-769f-42a4-bd82-3dac80ed345c admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:47.458566 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf2cb4-769f-42a4-bd82-3dac80ed345c admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:47.465779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f549d-51c2-4662-8c78-21c87274bc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:47.466412 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f549d-51c2-4662-8c78-21c87274bc0e admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:47.466726 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f549d-51c2-4662-8c78-21c87274bc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:47.466987 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f549d-51c2-4662-8c78-21c87274bc0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:47.467712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f549d-51c2-4662-8c78-21c87274bc0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:47.469880 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f549d-51c2-4662-8c78-21c87274bc0e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:47.510371 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1295/2614] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:18:47 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:47.511512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d4bf9-0f4a-49a4-bdda-85a456414cbf None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:47.523500 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d4bf9-0f4a-49a4-bdda-85a456414cbf None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:47.583632 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bafa55-5ad0-444f-bcee-3ce76c168221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:47.584256 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bafa55-5ad0-444f-bcee-3ce76c168221 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:47.584530 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bafa55-5ad0-444f-bcee-3ce76c168221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:47.584773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bafa55-5ad0-444f-bcee-3ce76c168221 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:47.585423 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bafa55-5ad0-444f-bcee-3ce76c168221 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:47.587870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bafa55-5ad0-444f-bcee-3ce76c168221 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:47.610204 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1320/2615] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:18:47 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:47.614276 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f549d-51c2-4662-8c78-21c87274bc0e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:47.624871 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f549d-51c2-4662-8c78-21c87274bc0e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:47.629751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92e52f-d019-435a-9aa4-a210af81cf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:47.630364 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92e52f-d019-435a-9aa4-a210af81cf04 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:47.630641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92e52f-d019-435a-9aa4-a210af81cf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:47.630911 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92e52f-d019-435a-9aa4-a210af81cf04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:47.631610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92e52f-d019-435a-9aa4-a210af81cf04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:47.633264 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92e52f-d019-435a-9aa4-a210af81cf04 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:47.645596 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1296/2616] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:47.649604 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bafa55-5ad0-444f-bcee-3ce76c168221 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:47.660321 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bafa55-5ad0-444f-bcee-3ce76c168221 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:47.665192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ef6cc-43e2-431a-abae-9c0480d8dd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:47.665892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ef6cc-43e2-431a-abae-9c0480d8dd3a admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:47.666236 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ef6cc-43e2-431a-abae-9c0480d8dd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:47.666571 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ef6cc-43e2-431a-abae-9c0480d8dd3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:47.667214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ef6cc-43e2-431a-abae-9c0480d8dd3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:47.668937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ef6cc-43e2-431a-abae-9c0480d8dd3a admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:47.675115 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:47.681581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1321/2617] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:18:47.686174 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92e52f-d019-435a-9aa4-a210af81cf04 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:47.696596 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92e52f-d019-435a-9aa4-a210af81cf04 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:47.702055 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807281c-8ac0-4b74-b97c-f9424e8b9270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:47.702730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807281c-8ac0-4b74-b97c-f9424e8b9270 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:47.703010 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807281c-8ac0-4b74-b97c-f9424e8b9270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:47.703286 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807281c-8ac0-4b74-b97c-f9424e8b9270 admin admin] PATH_INFO: `/v3/projects/dbf6a493bf5e4fcb802496d906a89425/users/52a9a4808b8b45688e8cf7bf76367e41/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:47.718864 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807281c-8ac0-4b74-b97c-f9424e8b9270 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf6a493bf5e4fcb802496d906a89425, user_id=52a9a4808b8b45688e8cf7bf76367e41, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:47.721738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807281c-8ac0-4b74-b97c-f9424e8b9270 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:47.736090 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1297/2618] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:47 2022] PUT /identity/v3/projects/dbf6a493bf5e4fcb802496d906a89425/users/52a9a4808b8b45688e8cf7bf76367e41/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:47.746579 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d3fe1-44c4-41e9-99c6-57cade0b6346 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:47.746864 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d3fe1-44c4-41e9-99c6-57cade0b6346 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:47.747138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d3fe1-44c4-41e9-99c6-57cade0b6346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:47.775741 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-8d8d3fe1-44c4-41e9-99c6-57cade0b6346 None None] MFA Rules not processed for user `52a9a4808b8b45688e8cf7bf76367e41`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:47.776954 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8d8d3fe1-44c4-41e9-99c6-57cade0b6346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:47.860519 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1322/2619] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:18:47.865286 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807281c-8ac0-4b74-b97c-f9424e8b9270 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:47.909506 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:47.950076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807281c-8ac0-4b74-b97c-f9424e8b9270 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:47.955712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983afa9-4424-4c4a-9e27-7627bc660b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:47.956589 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983afa9-4424-4c4a-9e27-7627bc660b12 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:47.957478 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983afa9-4424-4c4a-9e27-7627bc660b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:47.959707 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983afa9-4424-4c4a-9e27-7627bc660b12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:47.960406 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983afa9-4424-4c4a-9e27-7627bc660b12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:47.962385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983afa9-4424-4c4a-9e27-7627bc660b12 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:47.999843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1298/2620] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:18:47 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:48.001059 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d3fe1-44c4-41e9-99c6-57cade0b6346 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:48.012027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d3fe1-44c4-41e9-99c6-57cade0b6346 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:48.075903 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2be4b-8aa1-40fb-b621-d0415b7b448e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:48.076532 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2be4b-8aa1-40fb-b621-d0415b7b448e admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:48.076833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2be4b-8aa1-40fb-b621-d0415b7b448e admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:48.077105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2be4b-8aa1-40fb-b621-d0415b7b448e admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:48.077747 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2be4b-8aa1-40fb-b621-d0415b7b448e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:48.079718 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2be4b-8aa1-40fb-b621-d0415b7b448e admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:48.103158 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1323/2621] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:18:47 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:18:48.107309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983afa9-4424-4c4a-9e27-7627bc660b12 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:48.118318 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983afa9-4424-4c4a-9e27-7627bc660b12 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:48.123369 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0f5a3-abd7-4357-9af7-eeb65fee19d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:48.124099 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0f5a3-abd7-4357-9af7-eeb65fee19d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:48.124454 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0f5a3-abd7-4357-9af7-eeb65fee19d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:48.127304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0f5a3-abd7-4357-9af7-eeb65fee19d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:48.127304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0f5a3-abd7-4357-9af7-eeb65fee19d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:48.127304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0f5a3-abd7-4357-9af7-eeb65fee19d9 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:48.139927 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1299/2622] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:48.144987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2be4b-8aa1-40fb-b621-d0415b7b448e admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:48.157911 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2be4b-8aa1-40fb-b621-d0415b7b448e admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:48.163463 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6daa68-d169-40d8-ac66-291c32e88dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:48.164272 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6daa68-d169-40d8-ac66-291c32e88dce admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:48.164559 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6daa68-d169-40d8-ac66-291c32e88dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:48.164829 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6daa68-d169-40d8-ac66-291c32e88dce admin admin] PATH_INFO: `/v3/projects/cfcdd63869db43a281f8de59037a4d1a/users/23f88dc8e311452a8325e3a122a5ea98/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:48.181141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6daa68-d169-40d8-ac66-291c32e88dce admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfcdd63869db43a281f8de59037a4d1a, user_id=23f88dc8e311452a8325e3a122a5ea98, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:48.183871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6daa68-d169-40d8-ac66-291c32e88dce admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:48.200158 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1324/2623] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:48 2022] PUT /identity/v3/projects/cfcdd63869db43a281f8de59037a4d1a/users/23f88dc8e311452a8325e3a122a5ea98/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:48.204322 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0f5a3-abd7-4357-9af7-eeb65fee19d9 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:48.219682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0f5a3-abd7-4357-9af7-eeb65fee19d9 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:48.282825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cddd5-a855-42ba-8637-30826319306e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:48.283736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cddd5-a855-42ba-8637-30826319306e admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:48.284261 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cddd5-a855-42ba-8637-30826319306e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:48.284679 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cddd5-a855-42ba-8637-30826319306e admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:48.285335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7cddd5-a855-42ba-8637-30826319306e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:48.286978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7cddd5-a855-42ba-8637-30826319306e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:48.297247 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1300/2624] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:48.301427 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6daa68-d169-40d8-ac66-291c32e88dce admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:48.314876 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6daa68-d169-40d8-ac66-291c32e88dce admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:48.328949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a7e14-33c3-4b90-85c5-dc905477f619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:48.329807 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a7e14-33c3-4b90-85c5-dc905477f619 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:48.331179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a7e14-33c3-4b90-85c5-dc905477f619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:48.331574 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a7e14-33c3-4b90-85c5-dc905477f619 admin admin] PATH_INFO: `/v3/projects/cfcdd63869db43a281f8de59037a4d1a/users/23f88dc8e311452a8325e3a122a5ea98/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:48.333627 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a7e14-33c3-4b90-85c5-dc905477f619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfcdd63869db43a281f8de59037a4d1a, user_id=23f88dc8e311452a8325e3a122a5ea98, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:48.336383 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a7e14-33c3-4b90-85c5-dc905477f619 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:48.348091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1325/2625] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:48 2022] PUT /identity/v3/projects/cfcdd63869db43a281f8de59037a4d1a/users/23f88dc8e311452a8325e3a122a5ea98/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:18:48.358539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07259ccb-fa3e-4162-bd29-537dc3419f13 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:48.358822 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07259ccb-fa3e-4162-bd29-537dc3419f13 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:48.359095 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07259ccb-fa3e-4162-bd29-537dc3419f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:48.389058 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-07259ccb-fa3e-4162-bd29-537dc3419f13 None None] MFA Rules not processed for user `23f88dc8e311452a8325e3a122a5ea98`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:48.390331 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-07259ccb-fa3e-4162-bd29-537dc3419f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:48.463173 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1301/2626] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:48.474192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a7e14-33c3-4b90-85c5-dc905477f619 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:48.476028 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e75a7e14-33c3-4b90-85c5-dc905477f619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:48.524184 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:48.546041 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a7e14-33c3-4b90-85c5-dc905477f619 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:48.550500 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406b2d6-9f09-4827-86da-1b027c149aff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:48.551121 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406b2d6-9f09-4827-86da-1b027c149aff service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:48.551397 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406b2d6-9f09-4827-86da-1b027c149aff service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:48.551832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406b2d6-9f09-4827-86da-1b027c149aff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:48.562426 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406b2d6-9f09-4827-86da-1b027c149aff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:48.563901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406b2d6-9f09-4827-86da-1b027c149aff service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:48.573854 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1326/2627] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:18:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:18:48.619454 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07259ccb-fa3e-4162-bd29-537dc3419f13 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:48.620979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-07259ccb-fa3e-4162-bd29-537dc3419f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:48.633740 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07259ccb-fa3e-4162-bd29-537dc3419f13 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:48.637302 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd55fda-4a66-42ca-9f2b-676845d739d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:48.637910 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd55fda-4a66-42ca-9f2b-676845d739d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:48.638185 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd55fda-4a66-42ca-9f2b-676845d739d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:48.638479 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd55fda-4a66-42ca-9f2b-676845d739d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:48.665949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:48.705924 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd55fda-4a66-42ca-9f2b-676845d739d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:48.707405 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd55fda-4a66-42ca-9f2b-676845d739d0 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:48.716815 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1302/2628] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:18:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:18:51.668234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406b2d6-9f09-4827-86da-1b027c149aff service cinder] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:51.736189 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406b2d6-9f09-4827-86da-1b027c149aff service cinder] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:51.741186 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d89c8f-8568-46e9-bed2-dd609a6652b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:51.741890 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d89c8f-8568-46e9-bed2-dd609a6652b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:51.742153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d89c8f-8568-46e9-bed2-dd609a6652b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:51.742429 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d89c8f-8568-46e9-bed2-dd609a6652b9 admin admin] PATH_INFO: `/v3/users/de88d4b8b83043a19909f3c7c41b8287` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:51.743627 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d89c8f-8568-46e9-bed2-dd609a6652b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de88d4b8b83043a19909f3c7c41b8287)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:51.745581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d89c8f-8568-46e9-bed2-dd609a6652b9 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:51.796180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e5d89c8f-8568-46e9-bed2-dd609a6652b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de88d4b8b83043a19909f3c7c41b8287'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:51.796849 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e5d89c8f-8568-46e9-bed2-dd609a6652b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de88d4b8b83043a19909f3c7c41b8287'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:51.797432 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e5d89c8f-8568-46e9-bed2-dd609a6652b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de88d4b8b83043a19909f3c7c41b8287'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:51.805745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e5d89c8f-8568-46e9-bed2-dd609a6652b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de88d4b8b83043a19909f3c7c41b8287'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:51.819258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-e5d89c8f-8568-46e9-bed2-dd609a6652b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de88d4b8b83043a19909f3c7c41b8287'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:51.834611 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1327/2629] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:18:51 2022] DELETE /identity/v3/users/de88d4b8b83043a19909f3c7c41b8287 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:51.839655 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215271ea-e2e4-4357-ae8a-721bdb6c3bec None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:51.840141 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215271ea-e2e4-4357-ae8a-721bdb6c3bec None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:51.840340 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215271ea-e2e4-4357-ae8a-721bdb6c3bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:51.861616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-215271ea-e2e4-4357-ae8a-721bdb6c3bec None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:51.862648 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-215271ea-e2e4-4357-ae8a-721bdb6c3bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:51.923805 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1303/2630] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:51.933679 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d89c8f-8568-46e9-bed2-dd609a6652b9 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:51.935110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e5d89c8f-8568-46e9-bed2-dd609a6652b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:51.993866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d89c8f-8568-46e9-bed2-dd609a6652b9 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:51.997755 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231b4e3-57f4-4c99-bb9d-378d6012c6da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:51.998367 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231b4e3-57f4-4c99-bb9d-378d6012c6da service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:51.998620 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231b4e3-57f4-4c99-bb9d-378d6012c6da service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:51.998882 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231b4e3-57f4-4c99-bb9d-378d6012c6da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:52.009007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6231b4e3-57f4-4c99-bb9d-378d6012c6da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:52.010497 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6231b4e3-57f4-4c99-bb9d-378d6012c6da service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:52.020968 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1328/2631] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:18:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:18:52.494199 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215271ea-e2e4-4357-ae8a-721bdb6c3bec None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:52.495724 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-215271ea-e2e4-4357-ae8a-721bdb6c3bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:52.507617 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215271ea-e2e4-4357-ae8a-721bdb6c3bec None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:52.512510 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b15ee-85ac-4882-9be2-14398bf1a155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:52.513150 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b15ee-85ac-4882-9be2-14398bf1a155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:52.513424 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b15ee-85ac-4882-9be2-14398bf1a155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:52.513695 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b15ee-85ac-4882-9be2-14398bf1a155 admin admin] PATH_INFO: `/v3/projects/a7c015ee4d634bc0842620a32509c961` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:52.514865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b15ee-85ac-4882-9be2-14398bf1a155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7c015ee4d634bc0842620a32509c961)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:52.516894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b15ee-85ac-4882-9be2-14398bf1a155 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:52.567042 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1f3b15ee-85ac-4882-9be2-14398bf1a155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7c015ee4d634bc0842620a32509c961', 'request_id': 'req-1f3b15ee-85ac-4882-9be2-14398bf1a155'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:52.573131 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1f3b15ee-85ac-4882-9be2-14398bf1a155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7c015ee4d634bc0842620a32509c961', 'request_id': 'req-1f3b15ee-85ac-4882-9be2-14398bf1a155'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:52.583512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1f3b15ee-85ac-4882-9be2-14398bf1a155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7c015ee4d634bc0842620a32509c961', 'request_id': 'req-1f3b15ee-85ac-4882-9be2-14398bf1a155'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:52.597088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1f3b15ee-85ac-4882-9be2-14398bf1a155 admin admin] The token cache is being invalidate because project a7c015ee4d634bc0842620a32509c961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:18:52.597362 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1f3b15ee-85ac-4882-9be2-14398bf1a155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:52.598753 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1f3b15ee-85ac-4882-9be2-14398bf1a155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:52.600411 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1304/2632] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:18:52 2022] DELETE /identity/v3/projects/a7c015ee4d634bc0842620a32509c961 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:52.604635 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231b4e3-57f4-4c99-bb9d-378d6012c6da service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:52.615247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231b4e3-57f4-4c99-bb9d-378d6012c6da service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:52.619458 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:52.681047 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e861193-b17c-4e6b-99a0-d67fd6319658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:52.681778 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e861193-b17c-4e6b-99a0-d67fd6319658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:52.681974 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e861193-b17c-4e6b-99a0-d67fd6319658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:52.682249 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e861193-b17c-4e6b-99a0-d67fd6319658 admin admin] PATH_INFO: `/v3/users/52a9a4808b8b45688e8cf7bf76367e41` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:52.686507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e861193-b17c-4e6b-99a0-d67fd6319658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52a9a4808b8b45688e8cf7bf76367e41)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:52.688544 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e861193-b17c-4e6b-99a0-d67fd6319658 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:52.745643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1e861193-b17c-4e6b-99a0-d67fd6319658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52a9a4808b8b45688e8cf7bf76367e41'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:52.746330 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1e861193-b17c-4e6b-99a0-d67fd6319658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52a9a4808b8b45688e8cf7bf76367e41'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:52.746932 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1e861193-b17c-4e6b-99a0-d67fd6319658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52a9a4808b8b45688e8cf7bf76367e41'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:52.754231 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1e861193-b17c-4e6b-99a0-d67fd6319658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52a9a4808b8b45688e8cf7bf76367e41'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:52.767005 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1e861193-b17c-4e6b-99a0-d67fd6319658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52a9a4808b8b45688e8cf7bf76367e41'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:52.782124 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1329/2633] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:18:52 2022] DELETE /identity/v3/users/52a9a4808b8b45688e8cf7bf76367e41 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:53.138410 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b15ee-85ac-4882-9be2-14398bf1a155 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:53.140049 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-1f3b15ee-85ac-4882-9be2-14398bf1a155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:53.154080 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b15ee-85ac-4882-9be2-14398bf1a155 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:53.203724 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:53.218782 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d53e3e5-6f4b-48fc-86d0-1433f9b5af67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:53.219437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53e3e5-6f4b-48fc-86d0-1433f9b5af67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:53.219728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53e3e5-6f4b-48fc-86d0-1433f9b5af67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:53.220008 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53e3e5-6f4b-48fc-86d0-1433f9b5af67 admin admin] PATH_INFO: `/v3/projects/dbf6a493bf5e4fcb802496d906a89425` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:53.221136 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d53e3e5-6f4b-48fc-86d0-1433f9b5af67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbf6a493bf5e4fcb802496d906a89425)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:53.223333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d53e3e5-6f4b-48fc-86d0-1433f9b5af67 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:53.274461 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-0d53e3e5-6f4b-48fc-86d0-1433f9b5af67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbf6a493bf5e4fcb802496d906a89425', 'request_id': 'req-0d53e3e5-6f4b-48fc-86d0-1433f9b5af67'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:53.281337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-0d53e3e5-6f4b-48fc-86d0-1433f9b5af67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbf6a493bf5e4fcb802496d906a89425', 'request_id': 'req-0d53e3e5-6f4b-48fc-86d0-1433f9b5af67'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:53.290416 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-0d53e3e5-6f4b-48fc-86d0-1433f9b5af67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbf6a493bf5e4fcb802496d906a89425', 'request_id': 'req-0d53e3e5-6f4b-48fc-86d0-1433f9b5af67'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:53.302574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-0d53e3e5-6f4b-48fc-86d0-1433f9b5af67 admin admin] The token cache is being invalidate because project dbf6a493bf5e4fcb802496d906a89425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:18:53.302851 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-0d53e3e5-6f4b-48fc-86d0-1433f9b5af67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:53.303513 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-0d53e3e5-6f4b-48fc-86d0-1433f9b5af67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:53.305085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1305/2634] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:18:53 2022] DELETE /identity/v3/projects/dbf6a493bf5e4fcb802496d906a89425 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:53.309242 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e861193-b17c-4e6b-99a0-d67fd6319658 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:53.310800 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-1e861193-b17c-4e6b-99a0-d67fd6319658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:53.324294 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e861193-b17c-4e6b-99a0-d67fd6319658 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:53.396801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e38c3-3b38-4ab9-bd43-48d9d48a4e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:53.397364 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e38c3-3b38-4ab9-bd43-48d9d48a4e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:53.397643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e38c3-3b38-4ab9-bd43-48d9d48a4e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:53.397915 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e38c3-3b38-4ab9-bd43-48d9d48a4e92 admin admin] PATH_INFO: `/v3/users/23f88dc8e311452a8325e3a122a5ea98` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:53.400207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e38c3-3b38-4ab9-bd43-48d9d48a4e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23f88dc8e311452a8325e3a122a5ea98)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:53.402465 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e38c3-3b38-4ab9-bd43-48d9d48a4e92 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:53.443516 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:53.467332 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-282e38c3-3b38-4ab9-bd43-48d9d48a4e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23f88dc8e311452a8325e3a122a5ea98'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:53.468135 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-282e38c3-3b38-4ab9-bd43-48d9d48a4e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23f88dc8e311452a8325e3a122a5ea98'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:53.468745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-282e38c3-3b38-4ab9-bd43-48d9d48a4e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23f88dc8e311452a8325e3a122a5ea98'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:53.476571 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-282e38c3-3b38-4ab9-bd43-48d9d48a4e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23f88dc8e311452a8325e3a122a5ea98'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:53.489281 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-282e38c3-3b38-4ab9-bd43-48d9d48a4e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23f88dc8e311452a8325e3a122a5ea98'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:53.503275 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1330/2635] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:18:53 2022] DELETE /identity/v3/users/23f88dc8e311452a8325e3a122a5ea98 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:53.792843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d53e3e5-6f4b-48fc-86d0-1433f9b5af67 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:53.794290 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-0d53e3e5-6f4b-48fc-86d0-1433f9b5af67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:53.809477 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d53e3e5-6f4b-48fc-86d0-1433f9b5af67 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:53.869850 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771d15e-825b-43e4-bb8b-95dece9c80d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:53.870524 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771d15e-825b-43e4-bb8b-95dece9c80d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:53.870819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771d15e-825b-43e4-bb8b-95dece9c80d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:53.871089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771d15e-825b-43e4-bb8b-95dece9c80d8 admin admin] PATH_INFO: `/v3/projects/cfcdd63869db43a281f8de59037a4d1a` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:53.872350 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771d15e-825b-43e4-bb8b-95dece9c80d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfcdd63869db43a281f8de59037a4d1a)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:53.874342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771d15e-825b-43e4-bb8b-95dece9c80d8 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:53.921239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f771d15e-825b-43e4-bb8b-95dece9c80d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfcdd63869db43a281f8de59037a4d1a', 'request_id': 'req-f771d15e-825b-43e4-bb8b-95dece9c80d8'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:53.926937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f771d15e-825b-43e4-bb8b-95dece9c80d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfcdd63869db43a281f8de59037a4d1a', 'request_id': 'req-f771d15e-825b-43e4-bb8b-95dece9c80d8'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:53.935507 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f771d15e-825b-43e4-bb8b-95dece9c80d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfcdd63869db43a281f8de59037a4d1a', 'request_id': 'req-f771d15e-825b-43e4-bb8b-95dece9c80d8'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:53.947057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f771d15e-825b-43e4-bb8b-95dece9c80d8 admin admin] The token cache is being invalidate because project cfcdd63869db43a281f8de59037a4d1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:18:53.947357 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f771d15e-825b-43e4-bb8b-95dece9c80d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:53.948031 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f771d15e-825b-43e4-bb8b-95dece9c80d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:53.949519 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1306/2636] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:18:53 2022] DELETE /identity/v3/projects/cfcdd63869db43a281f8de59037a4d1a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:53.955791 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa89ce6-9b2a-4e7d-bc17-637383f3513c None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:53.956326 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa89ce6-9b2a-4e7d-bc17-637383f3513c None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:53.956424 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa89ce6-9b2a-4e7d-bc17-637383f3513c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:53.976119 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-4aa89ce6-9b2a-4e7d-bc17-637383f3513c None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:53.977314 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4aa89ce6-9b2a-4e7d-bc17-637383f3513c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:54.051541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1331/2637] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:54.061204 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f27fe-ef86-414c-94d1-2e76a6e2e1c2 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:54.061491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f27fe-ef86-414c-94d1-2e76a6e2e1c2 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:54.061768 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f27fe-ef86-414c-94d1-2e76a6e2e1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:54.077415 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:54.083779 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-481f27fe-ef86-414c-94d1-2e76a6e2e1c2 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:54.085901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-481f27fe-ef86-414c-94d1-2e76a6e2e1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:54.095051 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1307/2638] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:54.099721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa89ce6-9b2a-4e7d-bc17-637383f3513c None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:54.109483 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa89ce6-9b2a-4e7d-bc17-637383f3513c None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:54.114772 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b840a03-f47f-48b5-9ac7-089c5e42427f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:54.115386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b840a03-f47f-48b5-9ac7-089c5e42427f admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:54.115703 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b840a03-f47f-48b5-9ac7-089c5e42427f admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:54.115958 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b840a03-f47f-48b5-9ac7-089c5e42427f admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:54.116610 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b840a03-f47f-48b5-9ac7-089c5e42427f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:54.116916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b840a03-f47f-48b5-9ac7-089c5e42427f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:54.118561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b840a03-f47f-48b5-9ac7-089c5e42427f admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:54.131122 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1332/2639] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:54.136470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f27fe-ef86-414c-94d1-2e76a6e2e1c2 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:54.146685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f27fe-ef86-414c-94d1-2e76a6e2e1c2 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:54.151725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7ad84-951f-4856-825f-c30aa075b55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:54.152330 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7ad84-951f-4856-825f-c30aa075b55c admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:54.152607 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7ad84-951f-4856-825f-c30aa075b55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:54.152877 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7ad84-951f-4856-825f-c30aa075b55c admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:54.153496 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7ad84-951f-4856-825f-c30aa075b55c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:54.155405 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7ad84-951f-4856-825f-c30aa075b55c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:54.185962 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1308/2640] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:18:54 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:54.190634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b840a03-f47f-48b5-9ac7-089c5e42427f admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:54.202665 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b840a03-f47f-48b5-9ac7-089c5e42427f admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:54.265414 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185d524-7737-4c29-a992-1816a18fd733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:54.266062 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185d524-7737-4c29-a992-1816a18fd733 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:54.266345 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185d524-7737-4c29-a992-1816a18fd733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:54.266621 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185d524-7737-4c29-a992-1816a18fd733 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:54.267256 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185d524-7737-4c29-a992-1816a18fd733 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:54.269437 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185d524-7737-4c29-a992-1816a18fd733 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:54.294408 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1333/2641] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:18:54 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:54.295631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7ad84-951f-4856-825f-c30aa075b55c admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:54.306744 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7ad84-951f-4856-825f-c30aa075b55c admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:54.311576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3441b-61cb-420d-b471-c3e8da22c111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:54.312178 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3441b-61cb-420d-b471-c3e8da22c111 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:54.312459 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3441b-61cb-420d-b471-c3e8da22c111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:54.312738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3441b-61cb-420d-b471-c3e8da22c111 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:54.313369 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef3441b-61cb-420d-b471-c3e8da22c111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:54.314983 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef3441b-61cb-420d-b471-c3e8da22c111 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:54.325400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1309/2642] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:18:54.331965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185d524-7737-4c29-a992-1816a18fd733 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:54.343744 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185d524-7737-4c29-a992-1816a18fd733 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:54.350062 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ef50e-3d01-48ba-a7f2-6314e39db126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:54.350660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ef50e-3d01-48ba-a7f2-6314e39db126 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:54.350940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ef50e-3d01-48ba-a7f2-6314e39db126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:54.351211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ef50e-3d01-48ba-a7f2-6314e39db126 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:54.351895 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ef50e-3d01-48ba-a7f2-6314e39db126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:54.353498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ef50e-3d01-48ba-a7f2-6314e39db126 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:54.370900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1334/2643] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:54.377392 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3441b-61cb-420d-b471-c3e8da22c111 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:54.388850 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3441b-61cb-420d-b471-c3e8da22c111 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:54.395214 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31869db-ee0c-4261-9824-a4db90508fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:54.395898 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31869db-ee0c-4261-9824-a4db90508fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:54.396180 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31869db-ee0c-4261-9824-a4db90508fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:54.396453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31869db-ee0c-4261-9824-a4db90508fdf admin admin] PATH_INFO: `/v3/projects/b878c4c9dec245f397139913ac09e90a/users/bc3478dae2e54002b26230eeab8b92d0/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:54.412283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31869db-ee0c-4261-9824-a4db90508fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b878c4c9dec245f397139913ac09e90a, user_id=bc3478dae2e54002b26230eeab8b92d0, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:54.415013 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31869db-ee0c-4261-9824-a4db90508fdf admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:54.432696 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1310/2644] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:54 2022] PUT /identity/v3/projects/b878c4c9dec245f397139913ac09e90a/users/bc3478dae2e54002b26230eeab8b92d0/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:54.444255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfa202-c730-4ba0-b0d9-51d194e367e4 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:54.444482 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfa202-c730-4ba0-b0d9-51d194e367e4 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:54.444757 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfa202-c730-4ba0-b0d9-51d194e367e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:54.466364 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:54.475809 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-37cfa202-c730-4ba0-b0d9-51d194e367e4 None None] MFA Rules not processed for user `bc3478dae2e54002b26230eeab8b92d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:54.477790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-37cfa202-c730-4ba0-b0d9-51d194e367e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:54.544569 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1335/2645] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:54.549400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31869db-ee0c-4261-9824-a4db90508fdf admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:54.615577 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31869db-ee0c-4261-9824-a4db90508fdf admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:54.620470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cadb89-17b3-49d5-a22e-7ea9302f5ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:54.621086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cadb89-17b3-49d5-a22e-7ea9302f5ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:54.621370 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cadb89-17b3-49d5-a22e-7ea9302f5ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:54.621647 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cadb89-17b3-49d5-a22e-7ea9302f5ad6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:54.622275 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cadb89-17b3-49d5-a22e-7ea9302f5ad6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:54.624281 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cadb89-17b3-49d5-a22e-7ea9302f5ad6 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:54.660427 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1311/2646] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:18:54 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:54.661831 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cfa202-c730-4ba0-b0d9-51d194e367e4 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:54.673138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cfa202-c730-4ba0-b0d9-51d194e367e4 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:54.733598 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7be1b-1a68-40ec-98a0-6c907e22de9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:54.734212 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7be1b-1a68-40ec-98a0-6c907e22de9f admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:54.734487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7be1b-1a68-40ec-98a0-6c907e22de9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:54.734757 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7be1b-1a68-40ec-98a0-6c907e22de9f admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:54.735376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7be1b-1a68-40ec-98a0-6c907e22de9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:54.737347 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7be1b-1a68-40ec-98a0-6c907e22de9f admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:54.765106 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1336/2647] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:18:54 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:54.769144 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cadb89-17b3-49d5-a22e-7ea9302f5ad6 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:54.782240 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cadb89-17b3-49d5-a22e-7ea9302f5ad6 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:54.788193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f99bcc-7e17-4a41-bcb3-73c987f19414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:54.788845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99bcc-7e17-4a41-bcb3-73c987f19414 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:54.789127 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99bcc-7e17-4a41-bcb3-73c987f19414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:54.789401 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99bcc-7e17-4a41-bcb3-73c987f19414 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:54.790055 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f99bcc-7e17-4a41-bcb3-73c987f19414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:54.791719 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f99bcc-7e17-4a41-bcb3-73c987f19414 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:54.803142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1312/2648] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:54.807436 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7be1b-1a68-40ec-98a0-6c907e22de9f admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:54.817663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7be1b-1a68-40ec-98a0-6c907e22de9f admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:54.822790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7040f2-2815-460f-850d-327f202f6df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:54.823601 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7040f2-2815-460f-850d-327f202f6df7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:54.823877 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7040f2-2815-460f-850d-327f202f6df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:54.824151 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7040f2-2815-460f-850d-327f202f6df7 admin admin] PATH_INFO: `/v3/projects/f54f5fa58c564a21b0f66685aadafb36/users/24abb85497e244c08855eb424cd11709/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:54.840906 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7040f2-2815-460f-850d-327f202f6df7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f54f5fa58c564a21b0f66685aadafb36, user_id=24abb85497e244c08855eb424cd11709, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:54.843632 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7040f2-2815-460f-850d-327f202f6df7 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:54.858252 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1337/2649] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:54 2022] PUT /identity/v3/projects/f54f5fa58c564a21b0f66685aadafb36/users/24abb85497e244c08855eb424cd11709/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:18:54.862654 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f99bcc-7e17-4a41-bcb3-73c987f19414 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:54.872882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f99bcc-7e17-4a41-bcb3-73c987f19414 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:54.930459 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324515e9-e2bd-48f9-a5d7-a5ad9f8707ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:54.931087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324515e9-e2bd-48f9-a5d7-a5ad9f8707ed admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:54.931390 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324515e9-e2bd-48f9-a5d7-a5ad9f8707ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:54.931704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324515e9-e2bd-48f9-a5d7-a5ad9f8707ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:54.932461 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324515e9-e2bd-48f9-a5d7-a5ad9f8707ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:54.934273 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324515e9-e2bd-48f9-a5d7-a5ad9f8707ed admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:54.947979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1313/2650] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:54.952566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7040f2-2815-460f-850d-327f202f6df7 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:54.963405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7040f2-2815-460f-850d-327f202f6df7 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:54.968211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5bba7-2375-429f-b8d4-abf50381a6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:54.968918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5bba7-2375-429f-b8d4-abf50381a6c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:54.969267 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5bba7-2375-429f-b8d4-abf50381a6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:54.969643 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5bba7-2375-429f-b8d4-abf50381a6c5 admin admin] PATH_INFO: `/v3/projects/f54f5fa58c564a21b0f66685aadafb36/users/24abb85497e244c08855eb424cd11709/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:54.971672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b5bba7-2375-429f-b8d4-abf50381a6c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f54f5fa58c564a21b0f66685aadafb36, user_id=24abb85497e244c08855eb424cd11709, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:54.974603 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b5bba7-2375-429f-b8d4-abf50381a6c5 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:54.985066 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1338/2651] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:54 2022] PUT /identity/v3/projects/f54f5fa58c564a21b0f66685aadafb36/users/24abb85497e244c08855eb424cd11709/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:54.995588 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438592d3-53df-41fd-bec0-8760c317c8b5 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:54.995904 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438592d3-53df-41fd-bec0-8760c317c8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:54.996182 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438592d3-53df-41fd-bec0-8760c317c8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:55.022816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:55.025965 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-438592d3-53df-41fd-bec0-8760c317c8b5 None None] MFA Rules not processed for user `24abb85497e244c08855eb424cd11709`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:55.028095 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-438592d3-53df-41fd-bec0-8760c317c8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:55.100897 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1314/2652] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:55.117163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5bba7-2375-429f-b8d4-abf50381a6c5 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:55.120436 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-58b5bba7-2375-429f-b8d4-abf50381a6c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:55.197469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5bba7-2375-429f-b8d4-abf50381a6c5 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:55.201336 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45046762-129d-473e-9d12-1d8e261f297b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:55.201947 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45046762-129d-473e-9d12-1d8e261f297b service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:55.202220 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45046762-129d-473e-9d12-1d8e261f297b service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:55.202486 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45046762-129d-473e-9d12-1d8e261f297b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:55.209527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:55.212465 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45046762-129d-473e-9d12-1d8e261f297b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:55.214792 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45046762-129d-473e-9d12-1d8e261f297b service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:55.226353 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1339/2653] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:18:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:18:55.817364 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438592d3-53df-41fd-bec0-8760c317c8b5 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:55.884578 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438592d3-53df-41fd-bec0-8760c317c8b5 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:55.889231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78ec47-34ee-4b75-b145-b566e5d3ac72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:55.889871 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78ec47-34ee-4b75-b145-b566e5d3ac72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:55.890153 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78ec47-34ee-4b75-b145-b566e5d3ac72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:55.890423 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78ec47-34ee-4b75-b145-b566e5d3ac72 admin admin] PATH_INFO: `/v3/users/bc3478dae2e54002b26230eeab8b92d0` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:55.891642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78ec47-34ee-4b75-b145-b566e5d3ac72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc3478dae2e54002b26230eeab8b92d0)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:55.893607 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78ec47-34ee-4b75-b145-b566e5d3ac72 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:55.941468 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-ce78ec47-34ee-4b75-b145-b566e5d3ac72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc3478dae2e54002b26230eeab8b92d0'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:55.942340 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-ce78ec47-34ee-4b75-b145-b566e5d3ac72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc3478dae2e54002b26230eeab8b92d0'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:55.942946 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-ce78ec47-34ee-4b75-b145-b566e5d3ac72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc3478dae2e54002b26230eeab8b92d0'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:55.950604 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-ce78ec47-34ee-4b75-b145-b566e5d3ac72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc3478dae2e54002b26230eeab8b92d0'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:55.963559 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-ce78ec47-34ee-4b75-b145-b566e5d3ac72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc3478dae2e54002b26230eeab8b92d0'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:55.978103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1315/2654] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:18:55 2022] DELETE /identity/v3/users/bc3478dae2e54002b26230eeab8b92d0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:55.984877 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b7d2f-6a18-492c-923e-aebf438a0e35 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:55.985357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b7d2f-6a18-492c-923e-aebf438a0e35 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:55.985456 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b7d2f-6a18-492c-923e-aebf438a0e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:56.011588 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-8f1b7d2f-6a18-492c-923e-aebf438a0e35 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:56.013161 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8f1b7d2f-6a18-492c-923e-aebf438a0e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:56.084956 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1340/2655] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:56.095647 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78ec47-34ee-4b75-b145-b566e5d3ac72 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:56.097294 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ce78ec47-34ee-4b75-b145-b566e5d3ac72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:56.118529 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:56.158981 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78ec47-34ee-4b75-b145-b566e5d3ac72 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:56.163720 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9374b-88ad-4a8e-a044-66f91a472716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:56.164343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9374b-88ad-4a8e-a044-66f91a472716 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:56.164597 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9374b-88ad-4a8e-a044-66f91a472716 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:56.164864 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9374b-88ad-4a8e-a044-66f91a472716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:56.174047 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c9374b-88ad-4a8e-a044-66f91a472716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:56.175516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c9374b-88ad-4a8e-a044-66f91a472716 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:56.184964 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1316/2656] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:18:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:18:56.466777 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b7d2f-6a18-492c-923e-aebf438a0e35 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:56.468427 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8f1b7d2f-6a18-492c-923e-aebf438a0e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:56.481283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b7d2f-6a18-492c-923e-aebf438a0e35 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:56.485920 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8e187-b397-4230-8757-21b70737e6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:56.486560 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8e187-b397-4230-8757-21b70737e6e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:56.486834 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8e187-b397-4230-8757-21b70737e6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:56.487107 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8e187-b397-4230-8757-21b70737e6e6 admin admin] PATH_INFO: `/v3/projects/b878c4c9dec245f397139913ac09e90a` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:56.488432 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f8e187-b397-4230-8757-21b70737e6e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b878c4c9dec245f397139913ac09e90a)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:56.490581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f8e187-b397-4230-8757-21b70737e6e6 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:56.542714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-88f8e187-b397-4230-8757-21b70737e6e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b878c4c9dec245f397139913ac09e90a', 'request_id': 'req-88f8e187-b397-4230-8757-21b70737e6e6'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:56.548498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-88f8e187-b397-4230-8757-21b70737e6e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b878c4c9dec245f397139913ac09e90a', 'request_id': 'req-88f8e187-b397-4230-8757-21b70737e6e6'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:56.557018 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-88f8e187-b397-4230-8757-21b70737e6e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b878c4c9dec245f397139913ac09e90a', 'request_id': 'req-88f8e187-b397-4230-8757-21b70737e6e6'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:56.568898 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-88f8e187-b397-4230-8757-21b70737e6e6 admin admin] The token cache is being invalidate because project b878c4c9dec245f397139913ac09e90a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:18:56.569190 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-88f8e187-b397-4230-8757-21b70737e6e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:56.569824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-88f8e187-b397-4230-8757-21b70737e6e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:56.571363 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1341/2657] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:18:56 2022] DELETE /identity/v3/projects/b878c4c9dec245f397139913ac09e90a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:56.575514 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9374b-88ad-4a8e-a044-66f91a472716 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:56.585901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9374b-88ad-4a8e-a044-66f91a472716 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:56.649094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb235a65-1ab3-4b00-a101-a45f22278420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:56.649530 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb235a65-1ab3-4b00-a101-a45f22278420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:56.649811 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb235a65-1ab3-4b00-a101-a45f22278420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:56.650088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb235a65-1ab3-4b00-a101-a45f22278420 admin admin] PATH_INFO: `/v3/users/24abb85497e244c08855eb424cd11709` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:56.654403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb235a65-1ab3-4b00-a101-a45f22278420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24abb85497e244c08855eb424cd11709)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:56.654403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb235a65-1ab3-4b00-a101-a45f22278420 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:56.703100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-fb235a65-1ab3-4b00-a101-a45f22278420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24abb85497e244c08855eb424cd11709'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:56.703771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-fb235a65-1ab3-4b00-a101-a45f22278420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24abb85497e244c08855eb424cd11709'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:56.704366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-fb235a65-1ab3-4b00-a101-a45f22278420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24abb85497e244c08855eb424cd11709'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:56.711265 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-fb235a65-1ab3-4b00-a101-a45f22278420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24abb85497e244c08855eb424cd11709'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:56.726363 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-fb235a65-1ab3-4b00-a101-a45f22278420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24abb85497e244c08855eb424cd11709'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:56.752038 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1317/2658] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:18:56 2022] DELETE /identity/v3/users/24abb85497e244c08855eb424cd11709 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:57.104876 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8e187-b397-4230-8757-21b70737e6e6 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:57.106747 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-88f8e187-b397-4230-8757-21b70737e6e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:57.122891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8e187-b397-4230-8757-21b70737e6e6 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:57.202552 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccf78e-07b7-42cd-a3fa-e2821a749c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:57.203618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccf78e-07b7-42cd-a3fa-e2821a749c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:57.203885 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccf78e-07b7-42cd-a3fa-e2821a749c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:57.204158 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccf78e-07b7-42cd-a3fa-e2821a749c80 admin admin] PATH_INFO: `/v3/projects/f54f5fa58c564a21b0f66685aadafb36` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:57.205296 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccf78e-07b7-42cd-a3fa-e2821a749c80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f54f5fa58c564a21b0f66685aadafb36)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:57.207642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccf78e-07b7-42cd-a3fa-e2821a749c80 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:57.236334 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:57.265171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-36ccf78e-07b7-42cd-a3fa-e2821a749c80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f54f5fa58c564a21b0f66685aadafb36', 'request_id': 'req-36ccf78e-07b7-42cd-a3fa-e2821a749c80'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:57.271437 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-36ccf78e-07b7-42cd-a3fa-e2821a749c80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f54f5fa58c564a21b0f66685aadafb36', 'request_id': 'req-36ccf78e-07b7-42cd-a3fa-e2821a749c80'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:57.279930 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-36ccf78e-07b7-42cd-a3fa-e2821a749c80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f54f5fa58c564a21b0f66685aadafb36', 'request_id': 'req-36ccf78e-07b7-42cd-a3fa-e2821a749c80'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:57.291807 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-36ccf78e-07b7-42cd-a3fa-e2821a749c80 admin admin] The token cache is being invalidate because project f54f5fa58c564a21b0f66685aadafb36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:18:57.292125 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-36ccf78e-07b7-42cd-a3fa-e2821a749c80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:57.292762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-36ccf78e-07b7-42cd-a3fa-e2821a749c80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:18:57.294466 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1342/2659] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:18:57 2022] DELETE /identity/v3/projects/f54f5fa58c564a21b0f66685aadafb36 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:57.299228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c982d8-7b7d-4768-8ccf-65b3c2146f01 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:57.299725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c982d8-7b7d-4768-8ccf-65b3c2146f01 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:57.299833 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c982d8-7b7d-4768-8ccf-65b3c2146f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:57.320778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-61c982d8-7b7d-4768-8ccf-65b3c2146f01 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:57.322016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-61c982d8-7b7d-4768-8ccf-65b3c2146f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:57.358171 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:57.387142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1318/2660] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:57.464246 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f576b-dec2-4633-8648-817542a4aa78 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:57.464623 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f576b-dec2-4633-8648-817542a4aa78 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:57.464990 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f576b-dec2-4633-8648-817542a4aa78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:57.490337 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-660f576b-dec2-4633-8648-817542a4aa78 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:57.491604 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-660f576b-dec2-4633-8648-817542a4aa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:57.501452 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1343/2661] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:57.506658 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c982d8-7b7d-4768-8ccf-65b3c2146f01 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:57.517207 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c982d8-7b7d-4768-8ccf-65b3c2146f01 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:57.523079 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d677258-17d1-4ae9-8dec-a14dc5d249f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:57.523725 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d677258-17d1-4ae9-8dec-a14dc5d249f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:57.524000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d677258-17d1-4ae9-8dec-a14dc5d249f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:57.524267 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d677258-17d1-4ae9-8dec-a14dc5d249f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:57.524923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d677258-17d1-4ae9-8dec-a14dc5d249f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:18:57.525220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d677258-17d1-4ae9-8dec-a14dc5d249f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:57.526871 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d677258-17d1-4ae9-8dec-a14dc5d249f3 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:57.539483 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1319/2662] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:18:57.544211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f576b-dec2-4633-8648-817542a4aa78 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:57.553846 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f576b-dec2-4633-8648-817542a4aa78 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:57.558969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c13ad6-4a1c-4eab-9456-386ab88437ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:57.559585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c13ad6-4a1c-4eab-9456-386ab88437ac admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:57.559860 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c13ad6-4a1c-4eab-9456-386ab88437ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:57.560126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c13ad6-4a1c-4eab-9456-386ab88437ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:57.560756 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c13ad6-4a1c-4eab-9456-386ab88437ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:57.562642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c13ad6-4a1c-4eab-9456-386ab88437ac admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:57.593472 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1344/2663] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:18:57 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:57.598205 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d677258-17d1-4ae9-8dec-a14dc5d249f3 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:57.608685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d677258-17d1-4ae9-8dec-a14dc5d249f3 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:57.675156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abfb8f0-c5d5-4f51-b73a-3cf46d44ae9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:57.675949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abfb8f0-c5d5-4f51-b73a-3cf46d44ae9d admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:57.676225 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abfb8f0-c5d5-4f51-b73a-3cf46d44ae9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:57.676501 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abfb8f0-c5d5-4f51-b73a-3cf46d44ae9d admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:57.677137 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abfb8f0-c5d5-4f51-b73a-3cf46d44ae9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:57.679100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abfb8f0-c5d5-4f51-b73a-3cf46d44ae9d admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:57.713394 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1320/2664] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:18:57 2022] POST /identity/v3/users => generated 286 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:57.717131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c13ad6-4a1c-4eab-9456-386ab88437ac admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:57.728940 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c13ad6-4a1c-4eab-9456-386ab88437ac admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:57.733738 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd04289-abf9-416a-886c-36d84a74d9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:57.734654 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd04289-abf9-416a-886c-36d84a74d9ad admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:57.734773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd04289-abf9-416a-886c-36d84a74d9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:57.734945 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd04289-abf9-416a-886c-36d84a74d9ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:57.735668 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd04289-abf9-416a-886c-36d84a74d9ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:57.737257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd04289-abf9-416a-886c-36d84a74d9ad admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:57.751170 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1345/2665] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:18:57.755487 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abfb8f0-c5d5-4f51-b73a-3cf46d44ae9d admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:57.767035 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abfb8f0-c5d5-4f51-b73a-3cf46d44ae9d admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:57.772459 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587823ad-0841-4a45-98f9-3489545570f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:57.773179 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587823ad-0841-4a45-98f9-3489545570f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:57.773458 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587823ad-0841-4a45-98f9-3489545570f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:57.773728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587823ad-0841-4a45-98f9-3489545570f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:57.774400 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587823ad-0841-4a45-98f9-3489545570f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:57.776749 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587823ad-0841-4a45-98f9-3489545570f3 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:57.793626 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1321/2666] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:57.798325 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd04289-abf9-416a-886c-36d84a74d9ad admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:57.809874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd04289-abf9-416a-886c-36d84a74d9ad admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:57.815078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaeb4b5-16f2-40e7-b254-6b944b64fbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:57.815758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeb4b5-16f2-40e7-b254-6b944b64fbb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:57.816029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeb4b5-16f2-40e7-b254-6b944b64fbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:57.816297 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeb4b5-16f2-40e7-b254-6b944b64fbb4 admin admin] PATH_INFO: `/v3/projects/dd421691274c459faf51f0d534ceae15/users/747afb1be76c40578515dd2bb31ab097/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:57.838515 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaeb4b5-16f2-40e7-b254-6b944b64fbb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd421691274c459faf51f0d534ceae15, user_id=747afb1be76c40578515dd2bb31ab097, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:57.845804 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaeb4b5-16f2-40e7-b254-6b944b64fbb4 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:57.867775 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1346/2667] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:57 2022] PUT /identity/v3/projects/dd421691274c459faf51f0d534ceae15/users/747afb1be76c40578515dd2bb31ab097/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:57.873084 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f88bdf-9484-40ff-9522-d91192ffd87c None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:57.873373 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f88bdf-9484-40ff-9522-d91192ffd87c None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:57.873649 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f88bdf-9484-40ff-9522-d91192ffd87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:57.893395 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-e5f88bdf-9484-40ff-9522-d91192ffd87c None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:57.894858 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e5f88bdf-9484-40ff-9522-d91192ffd87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:57.958551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1322/2668] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:57.969061 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaeb4b5-16f2-40e7-b254-6b944b64fbb4 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:57.971254 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-caaeb4b5-16f2-40e7-b254-6b944b64fbb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:58.032412 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaeb4b5-16f2-40e7-b254-6b944b64fbb4 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:58.036773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6cc21e-49ad-4457-bd76-384e79eb9538 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:58.037389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cc21e-49ad-4457-bd76-384e79eb9538 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:58.037642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cc21e-49ad-4457-bd76-384e79eb9538 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:58.037946 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cc21e-49ad-4457-bd76-384e79eb9538 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:58.047785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6cc21e-49ad-4457-bd76-384e79eb9538 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:58.049354 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6cc21e-49ad-4457-bd76-384e79eb9538 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:58.063005 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1347/2669] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:18:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:18:58.801747 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823726c-7b18-4527-81d0-262ec7786126 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:58.802253 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823726c-7b18-4527-81d0-262ec7786126 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:58.802430 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823726c-7b18-4527-81d0-262ec7786126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:58.839389 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-8823726c-7b18-4527-81d0-262ec7786126 None None] MFA Rules not processed for user `747afb1be76c40578515dd2bb31ab097`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:18:58.840754 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8823726c-7b18-4527-81d0-262ec7786126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:18:58.880021 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:58.924368 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1323/2670] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:18:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:18:58.928870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6cc21e-49ad-4457-bd76-384e79eb9538 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:58.939213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6cc21e-49ad-4457-bd76-384e79eb9538 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:58.944632 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c23926-3951-479b-81cd-2a53c7fd464e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:58.945239 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c23926-3951-479b-81cd-2a53c7fd464e admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:58.945520 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c23926-3951-479b-81cd-2a53c7fd464e admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:58.945796 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c23926-3951-479b-81cd-2a53c7fd464e admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:58.946411 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c23926-3951-479b-81cd-2a53c7fd464e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:58.948380 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c23926-3951-479b-81cd-2a53c7fd464e admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:58.979527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1348/2671] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:18:58 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:18:58.983493 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823726c-7b18-4527-81d0-262ec7786126 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:58.996254 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823726c-7b18-4527-81d0-262ec7786126 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:59.074292 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c351931-673b-443b-b6ce-ee349f8ae4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:59.074928 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c351931-673b-443b-b6ce-ee349f8ae4cf admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:59.075242 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c351931-673b-443b-b6ce-ee349f8ae4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:59.075581 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c351931-673b-443b-b6ce-ee349f8ae4cf admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:59.076347 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c351931-673b-443b-b6ce-ee349f8ae4cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:59.079176 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c351931-673b-443b-b6ce-ee349f8ae4cf admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:59.106695 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1324/2672] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:18:58 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:18:59.110544 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c23926-3951-479b-81cd-2a53c7fd464e admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:59.121219 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c23926-3951-479b-81cd-2a53c7fd464e admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:59.125925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa0e4b-8491-4f6c-9c6e-b4e793fda13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:59.126522 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa0e4b-8491-4f6c-9c6e-b4e793fda13f admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:59.126799 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa0e4b-8491-4f6c-9c6e-b4e793fda13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:59.127068 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa0e4b-8491-4f6c-9c6e-b4e793fda13f admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:59.127746 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa0e4b-8491-4f6c-9c6e-b4e793fda13f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:59.129666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa0e4b-8491-4f6c-9c6e-b4e793fda13f admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:59.141059 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1349/2673] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:18:59.145601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c351931-673b-443b-b6ce-ee349f8ae4cf admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:59.161591 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c351931-673b-443b-b6ce-ee349f8ae4cf admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:59.166857 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327efae6-7bd8-44d2-ba91-1459658e5f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:59.167831 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327efae6-7bd8-44d2-ba91-1459658e5f90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:59.168107 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327efae6-7bd8-44d2-ba91-1459658e5f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:59.168375 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327efae6-7bd8-44d2-ba91-1459658e5f90 admin admin] PATH_INFO: `/v3/projects/d978691cf90a4acf9cf6e4c77db77dcc/users/e4d4ded54ada4a6ba3b2f7f867222412/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:59.184527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327efae6-7bd8-44d2-ba91-1459658e5f90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d978691cf90a4acf9cf6e4c77db77dcc, user_id=e4d4ded54ada4a6ba3b2f7f867222412, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:59.187249 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327efae6-7bd8-44d2-ba91-1459658e5f90 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:59.204048 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1325/2674] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:59 2022] PUT /identity/v3/projects/d978691cf90a4acf9cf6e4c77db77dcc/users/e4d4ded54ada4a6ba3b2f7f867222412/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:18:59.209186 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa0e4b-8491-4f6c-9c6e-b4e793fda13f admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:59.220063 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa0e4b-8491-4f6c-9c6e-b4e793fda13f admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:59.232832 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:18:59.288050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b19f4-fbff-45ff-a569-3faa58585a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:59.288678 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b19f4-fbff-45ff-a569-3faa58585a9a admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:59.288990 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b19f4-fbff-45ff-a569-3faa58585a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:59.289256 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b19f4-fbff-45ff-a569-3faa58585a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:59.289900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b19f4-fbff-45ff-a569-3faa58585a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:59.291541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b19f4-fbff-45ff-a569-3faa58585a9a admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:59.305763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1350/2675] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:18:59.309994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327efae6-7bd8-44d2-ba91-1459658e5f90 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:18:59.321151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327efae6-7bd8-44d2-ba91-1459658e5f90 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:18:59.326068 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b2028-2687-4eac-88ec-5bdc94476448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:18:59.326729 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b2028-2687-4eac-88ec-5bdc94476448 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:18:59.327003 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b2028-2687-4eac-88ec-5bdc94476448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:18:59.327275 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b2028-2687-4eac-88ec-5bdc94476448 admin admin] PATH_INFO: `/v3/projects/d978691cf90a4acf9cf6e4c77db77dcc/users/e4d4ded54ada4a6ba3b2f7f867222412/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:18:59.329228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b2028-2687-4eac-88ec-5bdc94476448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d978691cf90a4acf9cf6e4c77db77dcc, user_id=e4d4ded54ada4a6ba3b2f7f867222412, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:18:59.332130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b2028-2687-4eac-88ec-5bdc94476448 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:18:59.343015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1326/2676] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:18:59 2022] PUT /identity/v3/projects/d978691cf90a4acf9cf6e4c77db77dcc/users/e4d4ded54ada4a6ba3b2f7f867222412/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:19:00.147700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aba126-71bc-4c1b-8ed3-89034e9c29da None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:00.148700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aba126-71bc-4c1b-8ed3-89034e9c29da None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:00.149167 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aba126-71bc-4c1b-8ed3-89034e9c29da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:00.182808 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-60aba126-71bc-4c1b-8ed3-89034e9c29da None None] MFA Rules not processed for user `e4d4ded54ada4a6ba3b2f7f867222412`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:19:00.184166 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-60aba126-71bc-4c1b-8ed3-89034e9c29da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:00.275583 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1351/2677] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:19:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:19:00.279873 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b2028-2687-4eac-88ec-5bdc94476448 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:00.282283 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-847b2028-2687-4eac-88ec-5bdc94476448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:00.342183 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b2028-2687-4eac-88ec-5bdc94476448 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:00.345813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cfb04f-0df1-46ef-a29a-55033e659825 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:00.346434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfb04f-0df1-46ef-a29a-55033e659825 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:00.346710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfb04f-0df1-46ef-a29a-55033e659825 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:00.346981 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfb04f-0df1-46ef-a29a-55033e659825 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:00.389536 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:19:00.405418 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cfb04f-0df1-46ef-a29a-55033e659825 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:00.406913 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cfb04f-0df1-46ef-a29a-55033e659825 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:00.416016 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1327/2678] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:19:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:19:01.417563 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002353e-6994-4e46-873c-0c13689403bb None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:01.418016 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002353e-6994-4e46-873c-0c13689403bb None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:01.418135 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002353e-6994-4e46-873c-0c13689403bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:01.438441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-3002353e-6994-4e46-873c-0c13689403bb None None] MFA Rules not processed for user `b3f1ae88e29f4172a7db7393d517a893`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:19:01.439640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3002353e-6994-4e46-873c-0c13689403bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:01.502594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1352/2679] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 23 07:19:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:19:01.553062 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cfb04f-0df1-46ef-a29a-55033e659825 service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:01.554516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-97cfb04f-0df1-46ef-a29a-55033e659825 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:01.626273 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cfb04f-0df1-46ef-a29a-55033e659825 service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:01.631964 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d0a3c-1750-47bc-b9e5-0b9a908b88c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd88ec8b477443af95bff9af03471adf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:01.632597 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d0a3c-1750-47bc-b9e5-0b9a908b88c4 service swift] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:01.632880 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d0a3c-1750-47bc-b9e5-0b9a908b88c4 service swift] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:01.633148 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d0a3c-1750-47bc-b9e5-0b9a908b88c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:01.641680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d0a3c-1750-47bc-b9e5-0b9a908b88c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:01.643273 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d0a3c-1750-47bc-b9e5-0b9a908b88c4 service swift] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:01.651464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1328/2680] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 23 07:19:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 07:19:01.787764 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002353e-6994-4e46-873c-0c13689403bb None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:01.789413 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3002353e-6994-4e46-873c-0c13689403bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:01.802131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002353e-6994-4e46-873c-0c13689403bb None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:01.806047 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2766685-04fc-4fb8-a8ee-9b8be4ebdebc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:01.806696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2766685-04fc-4fb8-a8ee-9b8be4ebdebc service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:01.806981 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2766685-04fc-4fb8-a8ee-9b8be4ebdebc service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:01.807253 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2766685-04fc-4fb8-a8ee-9b8be4ebdebc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:01.817698 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2766685-04fc-4fb8-a8ee-9b8be4ebdebc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:01.819182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2766685-04fc-4fb8-a8ee-9b8be4ebdebc service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:01.830075 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1353/2681] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:19:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:19:11.648217 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=40 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:19:11.825400 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=87 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:19:16.071063 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d0a3c-1750-47bc-b9e5-0b9a908b88c4 service swift] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:16.074856 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395d9f9-4fab-45d4-b327-e31125f57241 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:16.075313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395d9f9-4fab-45d4-b327-e31125f57241 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:16.075568 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395d9f9-4fab-45d4-b327-e31125f57241 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:16.080245 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1354/2682] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 23 07:19:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:19:16.086197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960609f-8a4b-4041-a350-bfff84b128cc None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:16.086556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960609f-8a4b-4041-a350-bfff84b128cc None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:16.086943 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960609f-8a4b-4041-a350-bfff84b128cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:16.116413 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-8960609f-8a4b-4041-a350-bfff84b128cc None None] MFA Rules not processed for user `a8d2cd93042c4583b63afe629d47fce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:19:16.118366 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8960609f-8a4b-4041-a350-bfff84b128cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:16.146986 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d0a3c-1750-47bc-b9e5-0b9a908b88c4 service swift] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:16.151382 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1371114-ab0d-4d32-8be6-b36cc577fac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:16.152526 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1371114-ab0d-4d32-8be6-b36cc577fac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:16.152526 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1371114-ab0d-4d32-8be6-b36cc577fac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:16.152788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1371114-ab0d-4d32-8be6-b36cc577fac1 admin admin] PATH_INFO: `/v3/users/747afb1be76c40578515dd2bb31ab097` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:16.154115 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1371114-ab0d-4d32-8be6-b36cc577fac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=747afb1be76c40578515dd2bb31ab097)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:16.156410 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1371114-ab0d-4d32-8be6-b36cc577fac1 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:16.196995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1355/2683] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 23 07:19:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:19:16.211029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960609f-8a4b-4041-a350-bfff84b128cc None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:16.212508 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8960609f-8a4b-4041-a350-bfff84b128cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:16.225853 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f1371114-ab0d-4d32-8be6-b36cc577fac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '747afb1be76c40578515dd2bb31ab097'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:16.226557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f1371114-ab0d-4d32-8be6-b36cc577fac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '747afb1be76c40578515dd2bb31ab097'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:16.227142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f1371114-ab0d-4d32-8be6-b36cc577fac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '747afb1be76c40578515dd2bb31ab097'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:16.234548 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f1371114-ab0d-4d32-8be6-b36cc577fac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '747afb1be76c40578515dd2bb31ab097'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:16.246190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f1371114-ab0d-4d32-8be6-b36cc577fac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '747afb1be76c40578515dd2bb31ab097'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:16.261148 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1329/2684] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:19:16 2022] DELETE /identity/v3/users/747afb1be76c40578515dd2bb31ab097 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:19:16.276084 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960609f-8a4b-4041-a350-bfff84b128cc None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:16.328982 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6816d-29db-4e7f-a084-22d2e0d79ee5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57a5f04cb9674b86a6b4b9a472e1afc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:16.329651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6816d-29db-4e7f-a084-22d2e0d79ee5 service glance] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:16.329926 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6816d-29db-4e7f-a084-22d2e0d79ee5 service glance] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:16.330199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6816d-29db-4e7f-a084-22d2e0d79ee5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:16.332037 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-91c6816d-29db-4e7f-a084-22d2e0d79ee5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:16.392820 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:19:16.408511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6816d-29db-4e7f-a084-22d2e0d79ee5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:16.410037 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6816d-29db-4e7f-a084-22d2e0d79ee5 service glance] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:16.421155 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1356/2685] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:19:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:19:16.437548 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1371114-ab0d-4d32-8be6-b36cc577fac1 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:16.438977 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f1371114-ab0d-4d32-8be6-b36cc577fac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:16.520195 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1371114-ab0d-4d32-8be6-b36cc577fac1 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:16.525336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e1dad-e82c-4698-af17-41fe4e4f05aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:16.526133 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e1dad-e82c-4698-af17-41fe4e4f05aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:16.526415 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e1dad-e82c-4698-af17-41fe4e4f05aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:16.526722 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e1dad-e82c-4698-af17-41fe4e4f05aa admin admin] PATH_INFO: `/v3/projects/dd421691274c459faf51f0d534ceae15` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:16.527942 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71e1dad-e82c-4698-af17-41fe4e4f05aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd421691274c459faf51f0d534ceae15)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:16.529896 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71e1dad-e82c-4698-af17-41fe4e4f05aa admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:16.557198 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:19:16.584463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f71e1dad-e82c-4698-af17-41fe4e4f05aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd421691274c459faf51f0d534ceae15', 'request_id': 'req-f71e1dad-e82c-4698-af17-41fe4e4f05aa'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:16.590432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f71e1dad-e82c-4698-af17-41fe4e4f05aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd421691274c459faf51f0d534ceae15', 'request_id': 'req-f71e1dad-e82c-4698-af17-41fe4e4f05aa'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:16.598562 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f71e1dad-e82c-4698-af17-41fe4e4f05aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd421691274c459faf51f0d534ceae15', 'request_id': 'req-f71e1dad-e82c-4698-af17-41fe4e4f05aa'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:16.610918 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f71e1dad-e82c-4698-af17-41fe4e4f05aa admin admin] The token cache is being invalidate because project dd421691274c459faf51f0d534ceae15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:19:16.611206 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f71e1dad-e82c-4698-af17-41fe4e4f05aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:16.611893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f71e1dad-e82c-4698-af17-41fe4e4f05aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:16.613573 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1330/2686] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:19:16 2022] DELETE /identity/v3/projects/dd421691274c459faf51f0d534ceae15 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:19:16.617653 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6816d-29db-4e7f-a084-22d2e0d79ee5 service glance] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:16.628202 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6816d-29db-4e7f-a084-22d2e0d79ee5 service glance] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:16.692243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b6ebc-3de9-4af4-98f6-5afdb1bb2818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:16.692944 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b6ebc-3de9-4af4-98f6-5afdb1bb2818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:16.693230 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b6ebc-3de9-4af4-98f6-5afdb1bb2818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:16.693499 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b6ebc-3de9-4af4-98f6-5afdb1bb2818 admin admin] PATH_INFO: `/v3/users/e4d4ded54ada4a6ba3b2f7f867222412` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:16.694670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b6ebc-3de9-4af4-98f6-5afdb1bb2818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4d4ded54ada4a6ba3b2f7f867222412)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:16.697498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b6ebc-3de9-4af4-98f6-5afdb1bb2818 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:16.748666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-da9b6ebc-3de9-4af4-98f6-5afdb1bb2818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4d4ded54ada4a6ba3b2f7f867222412'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:16.749381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-da9b6ebc-3de9-4af4-98f6-5afdb1bb2818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4d4ded54ada4a6ba3b2f7f867222412'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:16.750048 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-da9b6ebc-3de9-4af4-98f6-5afdb1bb2818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4d4ded54ada4a6ba3b2f7f867222412'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:16.758025 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-da9b6ebc-3de9-4af4-98f6-5afdb1bb2818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4d4ded54ada4a6ba3b2f7f867222412'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:16.770883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-da9b6ebc-3de9-4af4-98f6-5afdb1bb2818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4d4ded54ada4a6ba3b2f7f867222412'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:16.786213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1357/2687] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:19:16 2022] DELETE /identity/v3/users/e4d4ded54ada4a6ba3b2f7f867222412 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:19:16.976159 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e1dad-e82c-4698-af17-41fe4e4f05aa admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:16.978201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f71e1dad-e82c-4698-af17-41fe4e4f05aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:17.007053 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e1dad-e82c-4698-af17-41fe4e4f05aa admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:17.076289 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ee16b-849a-4fe0-b5b7-606543fd2d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:17.076982 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ee16b-849a-4fe0-b5b7-606543fd2d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:17.077321 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ee16b-849a-4fe0-b5b7-606543fd2d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:17.077597 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ee16b-849a-4fe0-b5b7-606543fd2d10 admin admin] PATH_INFO: `/v3/projects/d978691cf90a4acf9cf6e4c77db77dcc` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:17.078785 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ee16b-849a-4fe0-b5b7-606543fd2d10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d978691cf90a4acf9cf6e4c77db77dcc)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:17.080787 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ee16b-849a-4fe0-b5b7-606543fd2d10 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:17.134244 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-791ee16b-849a-4fe0-b5b7-606543fd2d10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd978691cf90a4acf9cf6e4c77db77dcc', 'request_id': 'req-791ee16b-849a-4fe0-b5b7-606543fd2d10'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:17.139663 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-791ee16b-849a-4fe0-b5b7-606543fd2d10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd978691cf90a4acf9cf6e4c77db77dcc', 'request_id': 'req-791ee16b-849a-4fe0-b5b7-606543fd2d10'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:17.148011 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-791ee16b-849a-4fe0-b5b7-606543fd2d10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd978691cf90a4acf9cf6e4c77db77dcc', 'request_id': 'req-791ee16b-849a-4fe0-b5b7-606543fd2d10'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:17.159481 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-791ee16b-849a-4fe0-b5b7-606543fd2d10 admin admin] The token cache is being invalidate because project d978691cf90a4acf9cf6e4c77db77dcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:19:17.159774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-791ee16b-849a-4fe0-b5b7-606543fd2d10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:17.160455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-791ee16b-849a-4fe0-b5b7-606543fd2d10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:17.161965 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1331/2688] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:19:16 2022] DELETE /identity/v3/projects/d978691cf90a4acf9cf6e4c77db77dcc => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:19:17.166941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbe6c7-c202-4d56-a6aa-9fc34a4476d3 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:17.167404 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbe6c7-c202-4d56-a6aa-9fc34a4476d3 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:17.167571 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbe6c7-c202-4d56-a6aa-9fc34a4476d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:17.187133 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-94bbe6c7-c202-4d56-a6aa-9fc34a4476d3 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:19:17.188541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-94bbe6c7-c202-4d56-a6aa-9fc34a4476d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:17.253191 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1358/2689] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:19:17.261614 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72920fc-76d7-4af0-860b-8a4bcb83bdc6 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:17.261849 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72920fc-76d7-4af0-860b-8a4bcb83bdc6 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:17.262152 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72920fc-76d7-4af0-860b-8a4bcb83bdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:17.282313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-a72920fc-76d7-4af0-860b-8a4bcb83bdc6 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:19:17.283570 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-a72920fc-76d7-4af0-860b-8a4bcb83bdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:17.293006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1332/2690] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:19:17.297260 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbe6c7-c202-4d56-a6aa-9fc34a4476d3 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:17.308364 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbe6c7-c202-4d56-a6aa-9fc34a4476d3 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:17.313309 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fc876-d3c0-42da-b727-d84a42571376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:17.313995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fc876-d3c0-42da-b727-d84a42571376 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:17.314384 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fc876-d3c0-42da-b727-d84a42571376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:17.314746 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fc876-d3c0-42da-b727-d84a42571376 admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:17.315521 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726fc876-d3c0-42da-b727-d84a42571376 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:19:17.315901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726fc876-d3c0-42da-b727-d84a42571376 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:17.317762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726fc876-d3c0-42da-b727-d84a42571376 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:17.323260 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:19:17.339182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1359/2691] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:19:17.340429 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72920fc-76d7-4af0-860b-8a4bcb83bdc6 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:17.352736 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72920fc-76d7-4af0-860b-8a4bcb83bdc6 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:17.359798 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15195f5d-c00b-4276-abf0-fba51cf89b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:17.360976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15195f5d-c00b-4276-abf0-fba51cf89b41 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:17.361243 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15195f5d-c00b-4276-abf0-fba51cf89b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:17.361540 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15195f5d-c00b-4276-abf0-fba51cf89b41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:17.362169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15195f5d-c00b-4276-abf0-fba51cf89b41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:17.364640 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15195f5d-c00b-4276-abf0-fba51cf89b41 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:17.398483 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1333/2692] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:19:17 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:19:17.400244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fc876-d3c0-42da-b727-d84a42571376 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:17.411139 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fc876-d3c0-42da-b727-d84a42571376 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:17.477107 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c596ae-cac4-4964-8409-49467e365d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:17.477727 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c596ae-cac4-4964-8409-49467e365d66 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:17.478008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c596ae-cac4-4964-8409-49467e365d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:17.478282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c596ae-cac4-4964-8409-49467e365d66 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:17.478916 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c596ae-cac4-4964-8409-49467e365d66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:17.480879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c596ae-cac4-4964-8409-49467e365d66 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:17.505963 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1360/2693] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:19:17 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:19:17.507208 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15195f5d-c00b-4276-abf0-fba51cf89b41 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:17.518659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15195f5d-c00b-4276-abf0-fba51cf89b41 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:17.523595 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50c200-09f2-4805-b6c8-52d895d1e6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:17.524223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50c200-09f2-4805-b6c8-52d895d1e6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:17.524479 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50c200-09f2-4805-b6c8-52d895d1e6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:17.524750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50c200-09f2-4805-b6c8-52d895d1e6b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:17.525399 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50c200-09f2-4805-b6c8-52d895d1e6b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:17.527192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50c200-09f2-4805-b6c8-52d895d1e6b7 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:17.532740 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:19:17.541032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1334/2694] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:19:17.542225 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c596ae-cac4-4964-8409-49467e365d66 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:17.552969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c596ae-cac4-4964-8409-49467e365d66 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:17.557925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f43bc1-5a2c-4d4f-9766-38b7f0df6bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:17.558571 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f43bc1-5a2c-4d4f-9766-38b7f0df6bbb admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:17.558852 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f43bc1-5a2c-4d4f-9766-38b7f0df6bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:17.559126 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f43bc1-5a2c-4d4f-9766-38b7f0df6bbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:17.559809 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f43bc1-5a2c-4d4f-9766-38b7f0df6bbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:17.561420 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f43bc1-5a2c-4d4f-9766-38b7f0df6bbb admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:17.573285 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1361/2695] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:19:17.579316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50c200-09f2-4805-b6c8-52d895d1e6b7 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:17.590136 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50c200-09f2-4805-b6c8-52d895d1e6b7 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:17.594753 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b7ec5-a3ef-4fec-b559-3304109feb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:17.595531 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b7ec5-a3ef-4fec-b559-3304109feb6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:17.595812 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b7ec5-a3ef-4fec-b559-3304109feb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:17.596086 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b7ec5-a3ef-4fec-b559-3304109feb6a admin admin] PATH_INFO: `/v3/projects/f91899daf5a0458895d9f63011d8b4e1/users/2f0b8900d0ba467a967f867b17fc48c9/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:17.612483 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b7ec5-a3ef-4fec-b559-3304109feb6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f91899daf5a0458895d9f63011d8b4e1, user_id=2f0b8900d0ba467a967f867b17fc48c9, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:17.615236 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b7ec5-a3ef-4fec-b559-3304109feb6a admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:17.630522 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1335/2696] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:19:17 2022] PUT /identity/v3/projects/f91899daf5a0458895d9f63011d8b4e1/users/2f0b8900d0ba467a967f867b17fc48c9/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:19:17.641196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e42b3f-2f4e-492d-af86-28f23c6c3f72 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:17.641524 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e42b3f-2f4e-492d-af86-28f23c6c3f72 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:17.641799 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e42b3f-2f4e-492d-af86-28f23c6c3f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:17.677298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-86e42b3f-2f4e-492d-af86-28f23c6c3f72 None None] MFA Rules not processed for user `2f0b8900d0ba467a967f867b17fc48c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:19:17.678666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-86e42b3f-2f4e-492d-af86-28f23c6c3f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:17.748119 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1362/2697] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:19:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:19:17.759881 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b7ec5-a3ef-4fec-b559-3304109feb6a admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:17.761940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-7d2b7ec5-a3ef-4fec-b559-3304109feb6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:17.823615 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b7ec5-a3ef-4fec-b559-3304109feb6a admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:17.827312 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2c1da-5e9d-477d-8b0d-e6c337d70188 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:17.827979 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2c1da-5e9d-477d-8b0d-e6c337d70188 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:17.828261 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2c1da-5e9d-477d-8b0d-e6c337d70188 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:17.828536 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2c1da-5e9d-477d-8b0d-e6c337d70188 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:17.837909 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2c1da-5e9d-477d-8b0d-e6c337d70188 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:17.839895 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2c1da-5e9d-477d-8b0d-e6c337d70188 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:17.850442 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1336/2698] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:19:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:19:17.873733 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e42b3f-2f4e-492d-af86-28f23c6c3f72 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:17.946454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e42b3f-2f4e-492d-af86-28f23c6c3f72 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:17.951631 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1281112-e9d9-4025-8787-c4f3ad67fba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:17.952296 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1281112-e9d9-4025-8787-c4f3ad67fba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:17.952580 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1281112-e9d9-4025-8787-c4f3ad67fba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:17.952853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1281112-e9d9-4025-8787-c4f3ad67fba9 admin admin] PATH_INFO: `/v3/users/2f0b8900d0ba467a967f867b17fc48c9` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:17.954001 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1281112-e9d9-4025-8787-c4f3ad67fba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f0b8900d0ba467a967f867b17fc48c9)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:17.956071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1281112-e9d9-4025-8787-c4f3ad67fba9 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:17.961248 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:19:18.011662 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-b1281112-e9d9-4025-8787-c4f3ad67fba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f0b8900d0ba467a967f867b17fc48c9'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:18.012289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-b1281112-e9d9-4025-8787-c4f3ad67fba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f0b8900d0ba467a967f867b17fc48c9'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:18.012903 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-b1281112-e9d9-4025-8787-c4f3ad67fba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f0b8900d0ba467a967f867b17fc48c9'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:18.020634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-b1281112-e9d9-4025-8787-c4f3ad67fba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f0b8900d0ba467a967f867b17fc48c9'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:18.034527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-b1281112-e9d9-4025-8787-c4f3ad67fba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f0b8900d0ba467a967f867b17fc48c9'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:18.052375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1363/2699] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:19:17 2022] DELETE /identity/v3/users/2f0b8900d0ba467a967f867b17fc48c9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:19:18.056664 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8837435-7185-4222-858a-c8a0ba6eca1b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:18.056954 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8837435-7185-4222-858a-c8a0ba6eca1b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:18.057233 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8837435-7185-4222-858a-c8a0ba6eca1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:18.078241 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-f8837435-7185-4222-858a-c8a0ba6eca1b None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:19:18.079471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f8837435-7185-4222-858a-c8a0ba6eca1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:18.144039 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1337/2700] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:19:18.153309 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1281112-e9d9-4025-8787-c4f3ad67fba9 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:18.154903 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b1281112-e9d9-4025-8787-c4f3ad67fba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:18.348038 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1281112-e9d9-4025-8787-c4f3ad67fba9 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:18.352527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116faba7-486e-4477-b499-d60406b09d07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:18.353076 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116faba7-486e-4477-b499-d60406b09d07 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:18.353424 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116faba7-486e-4477-b499-d60406b09d07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:18.353707 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116faba7-486e-4477-b499-d60406b09d07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:18.363970 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116faba7-486e-4477-b499-d60406b09d07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:18.365427 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116faba7-486e-4477-b499-d60406b09d07 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:18.375485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1364/2701] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:19:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:19:18.665793 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8837435-7185-4222-858a-c8a0ba6eca1b None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:18.667773 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f8837435-7185-4222-858a-c8a0ba6eca1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:18.686127 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8837435-7185-4222-858a-c8a0ba6eca1b None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:18.692083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6b73b-1163-4b59-839a-0f598b24ff31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:18.692477 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6b73b-1163-4b59-839a-0f598b24ff31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:18.692864 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6b73b-1163-4b59-839a-0f598b24ff31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:18.693235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6b73b-1163-4b59-839a-0f598b24ff31 admin admin] PATH_INFO: `/v3/projects/f91899daf5a0458895d9f63011d8b4e1` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:18.694780 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e6b73b-1163-4b59-839a-0f598b24ff31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f91899daf5a0458895d9f63011d8b4e1)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:18.697420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e6b73b-1163-4b59-839a-0f598b24ff31 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:18.750326 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b8e6b73b-1163-4b59-839a-0f598b24ff31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f91899daf5a0458895d9f63011d8b4e1', 'request_id': 'req-b8e6b73b-1163-4b59-839a-0f598b24ff31'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:18.756488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b8e6b73b-1163-4b59-839a-0f598b24ff31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f91899daf5a0458895d9f63011d8b4e1', 'request_id': 'req-b8e6b73b-1163-4b59-839a-0f598b24ff31'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:18.764902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b8e6b73b-1163-4b59-839a-0f598b24ff31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f91899daf5a0458895d9f63011d8b4e1', 'request_id': 'req-b8e6b73b-1163-4b59-839a-0f598b24ff31'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:18.776586 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b8e6b73b-1163-4b59-839a-0f598b24ff31 admin admin] The token cache is being invalidate because project f91899daf5a0458895d9f63011d8b4e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:19:18.776872 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b8e6b73b-1163-4b59-839a-0f598b24ff31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:18.777517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-b8e6b73b-1163-4b59-839a-0f598b24ff31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:19:18.778991 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1338/2702] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:19:18 2022] DELETE /identity/v3/projects/f91899daf5a0458895d9f63011d8b4e1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:19:18.907391 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace2ba8-00c9-4917-b962-922d75f12b10 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:18.907794 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace2ba8-00c9-4917-b962-922d75f12b10 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:18.908073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace2ba8-00c9-4917-b962-922d75f12b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:18.930387 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-7ace2ba8-00c9-4917-b962-922d75f12b10 None None] MFA Rules not processed for user `b3f1ae88e29f4172a7db7393d517a893`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:19:18.931659 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-7ace2ba8-00c9-4917-b962-922d75f12b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:18.997584 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1365/2703] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 23 07:19:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:19:19.010717 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6b73b-1163-4b59-839a-0f598b24ff31 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:19.012217 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-b8e6b73b-1163-4b59-839a-0f598b24ff31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:19.017733 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:19:19.095260 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6b73b-1163-4b59-839a-0f598b24ff31 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:19.100812 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d33ad-a193-4237-9691-a91af8f63714 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd88ec8b477443af95bff9af03471adf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:19.101427 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d33ad-a193-4237-9691-a91af8f63714 service swift] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:19.101704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d33ad-a193-4237-9691-a91af8f63714 service swift] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:19.101976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d33ad-a193-4237-9691-a91af8f63714 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:19.110311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d33ad-a193-4237-9691-a91af8f63714 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:19.111958 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d33ad-a193-4237-9691-a91af8f63714 service swift] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:19.120309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1339/2704] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 23 07:19:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 07:19:28.990326 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=75,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:19:29.116951 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=31 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:19:40.692684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337ec9c-628c-48a9-99f7-5032894fa05f None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:40.693151 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337ec9c-628c-48a9-99f7-5032894fa05f None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:40.693255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337ec9c-628c-48a9-99f7-5032894fa05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:40.712826 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-6337ec9c-628c-48a9-99f7-5032894fa05f None None] MFA Rules not processed for user `b3f1ae88e29f4172a7db7393d517a893`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:19:40.714370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6337ec9c-628c-48a9-99f7-5032894fa05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:19:40.721697 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1366/2705] 10.222.0.17 () {58 vars in 967 bytes} [Tue Aug 23 07:19:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:19:40.738123 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d33ad-a193-4237-9691-a91af8f63714 service swift] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:19:40.752716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d33ad-a193-4237-9691-a91af8f63714 service swift] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:19:40.764193 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205fe7c2-0b59-44c3-b128-3be9766c0ea4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd88ec8b477443af95bff9af03471adf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:19:40.764804 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fe7c2-0b59-44c3-b128-3be9766c0ea4 service swift] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:19:40.765087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fe7c2-0b59-44c3-b128-3be9766c0ea4 service swift] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:19:40.765360 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fe7c2-0b59-44c3-b128-3be9766c0ea4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:19:40.774855 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205fe7c2-0b59-44c3-b128-3be9766c0ea4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:19:40.776488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205fe7c2-0b59-44c3-b128-3be9766c0ea4 service swift] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:19:40.784777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1340/2706] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 23 07:19:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 07:19:50.708121 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:19:50.781457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=6 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:20:15.056784 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337ec9c-628c-48a9-99f7-5032894fa05f None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:20:15.058469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6337ec9c-628c-48a9-99f7-5032894fa05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:20:15.145242 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337ec9c-628c-48a9-99f7-5032894fa05f None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:20:15.152412 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b3073-b4a4-48be-a80b-092015a29a8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:20:15.153026 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b3073-b4a4-48be-a80b-092015a29a8e service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:20:15.153283 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b3073-b4a4-48be-a80b-092015a29a8e service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:20:15.153551 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b3073-b4a4-48be-a80b-092015a29a8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:20:15.155456 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d32b3073-b4a4-48be-a80b-092015a29a8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:20:15.222687 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b3073-b4a4-48be-a80b-092015a29a8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:20:15.224364 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b3073-b4a4-48be-a80b-092015a29a8e service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:20:15.234093 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1367/2707] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:20:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:20:15.504057 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205fe7c2-0b59-44c3-b128-3be9766c0ea4 service swift] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:20:15.505753 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-205fe7c2-0b59-44c3-b128-3be9766c0ea4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:20:15.574222 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205fe7c2-0b59-44c3-b128-3be9766c0ea4 service swift] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:20:15.578174 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8760bc8-8489-4a82-824a-4930e400e558 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:20:15.578787 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8760bc8-8489-4a82-824a-4930e400e558 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:20:15.579064 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8760bc8-8489-4a82-824a-4930e400e558 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:20:15.579336 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8760bc8-8489-4a82-824a-4930e400e558 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:20:15.581196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d8760bc8-8489-4a82-824a-4930e400e558 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:20:15.646467 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8760bc8-8489-4a82-824a-4930e400e558 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:20:15.647965 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8760bc8-8489-4a82-824a-4930e400e558 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:20:15.657860 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1341/2708] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:20:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:20:22.395139 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e6cdd-da15-46c6-a868-011660193e28 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:20:22.395640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e6cdd-da15-46c6-a868-011660193e28 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:20:22.395758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e6cdd-da15-46c6-a868-011660193e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:20:22.429366 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-021e6cdd-da15-46c6-a868-011660193e28 None None] MFA Rules not processed for user `4fb28547f4b54c4097d2ffc3c533ca67`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:20:22.430702 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-021e6cdd-da15-46c6-a868-011660193e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:20:22.517389 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1368/2709] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:20:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:20:22.864132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8760bc8-8489-4a82-824a-4930e400e558 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:20:22.865636 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d8760bc8-8489-4a82-824a-4930e400e558 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:20:22.927085 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8760bc8-8489-4a82-824a-4930e400e558 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:20:22.933885 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f345b-ff3c-45ab-9461-a529c550c879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:20:22.934518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f345b-ff3c-45ab-9461-a529c550c879 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:20:22.934868 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f345b-ff3c-45ab-9461-a529c550c879 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:20:22.935140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f345b-ff3c-45ab-9461-a529c550c879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:20:22.936986 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8e5f345b-ff3c-45ab-9461-a529c550c879 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:20:22.949026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f345b-ff3c-45ab-9461-a529c550c879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:20:22.950960 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f345b-ff3c-45ab-9461-a529c550c879 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:20:22.963360 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1342/2710] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:20:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:20:22.979633 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e6cdd-da15-46c6-a868-011660193e28 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:20:22.981477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-021e6cdd-da15-46c6-a868-011660193e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:20:23.005277 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:20:23.045939 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e6cdd-da15-46c6-a868-011660193e28 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:20:23.051498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2479a65-9ce9-4b40-8531-0f616fcbd8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:20:23.052170 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2479a65-9ce9-4b40-8531-0f616fcbd8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:20:23.053039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2479a65-9ce9-4b40-8531-0f616fcbd8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:20:23.053210 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2479a65-9ce9-4b40-8531-0f616fcbd8dc admin admin] PATH_INFO: `/v3/users/c26232f042844a898c19d1f63152b428` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:20:23.054370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2479a65-9ce9-4b40-8531-0f616fcbd8dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c26232f042844a898c19d1f63152b428)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:20:23.056629 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2479a65-9ce9-4b40-8531-0f616fcbd8dc admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:20:23.114586 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d2479a65-9ce9-4b40-8531-0f616fcbd8dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c26232f042844a898c19d1f63152b428'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:23.115570 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d2479a65-9ce9-4b40-8531-0f616fcbd8dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c26232f042844a898c19d1f63152b428'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:23.116234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d2479a65-9ce9-4b40-8531-0f616fcbd8dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c26232f042844a898c19d1f63152b428'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:23.129630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d2479a65-9ce9-4b40-8531-0f616fcbd8dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c26232f042844a898c19d1f63152b428'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:23.153298 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-d2479a65-9ce9-4b40-8531-0f616fcbd8dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c26232f042844a898c19d1f63152b428'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:23.169386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1369/2711] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:20:22 2022] DELETE /identity/v3/users/c26232f042844a898c19d1f63152b428 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:20:23.377798 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f345b-ff3c-45ab-9461-a529c550c879 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:20:23.401457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f345b-ff3c-45ab-9461-a529c550c879 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:20:23.468445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876483b-1b51-4608-80f5-e74cba525a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:20:23.469091 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876483b-1b51-4608-80f5-e74cba525a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:20:23.469371 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876483b-1b51-4608-80f5-e74cba525a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:20:23.469646 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876483b-1b51-4608-80f5-e74cba525a45 admin admin] PATH_INFO: `/v3/projects/fd004facbaed41a7a54a3ed6a6a44305` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:20:23.470823 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876483b-1b51-4608-80f5-e74cba525a45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd004facbaed41a7a54a3ed6a6a44305)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:20:23.472803 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876483b-1b51-4608-80f5-e74cba525a45 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:20:23.483966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=100 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:20:23.526197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d876483b-1b51-4608-80f5-e74cba525a45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd004facbaed41a7a54a3ed6a6a44305', 'request_id': 'req-d876483b-1b51-4608-80f5-e74cba525a45'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:23.536420 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d876483b-1b51-4608-80f5-e74cba525a45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd004facbaed41a7a54a3ed6a6a44305', 'request_id': 'req-d876483b-1b51-4608-80f5-e74cba525a45'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:23.548409 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d876483b-1b51-4608-80f5-e74cba525a45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd004facbaed41a7a54a3ed6a6a44305', 'request_id': 'req-d876483b-1b51-4608-80f5-e74cba525a45'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:23.568291 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d876483b-1b51-4608-80f5-e74cba525a45 admin admin] The token cache is being invalidate because project fd004facbaed41a7a54a3ed6a6a44305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:20:23.568619 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d876483b-1b51-4608-80f5-e74cba525a45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:23.569498 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-d876483b-1b51-4608-80f5-e74cba525a45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:23.574975 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1343/2712] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:20:23 2022] DELETE /identity/v3/projects/fd004facbaed41a7a54a3ed6a6a44305 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:20:23.576213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2479a65-9ce9-4b40-8531-0f616fcbd8dc admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:20:23.577883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d2479a65-9ce9-4b40-8531-0f616fcbd8dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:20:23.593642 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2479a65-9ce9-4b40-8531-0f616fcbd8dc admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:20:23.676520 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858c2ec-7250-40d5-8b8b-8f52ad053a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:20:23.677254 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858c2ec-7250-40d5-8b8b-8f52ad053a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:20:23.677568 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858c2ec-7250-40d5-8b8b-8f52ad053a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:20:23.677857 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858c2ec-7250-40d5-8b8b-8f52ad053a04 admin admin] PATH_INFO: `/v3/users/c11e70f0fb9f49dbb4082a5fd35f13fb` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:20:23.679129 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858c2ec-7250-40d5-8b8b-8f52ad053a04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c11e70f0fb9f49dbb4082a5fd35f13fb)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:20:23.681310 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858c2ec-7250-40d5-8b8b-8f52ad053a04 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:20:23.735683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-9858c2ec-7250-40d5-8b8b-8f52ad053a04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c11e70f0fb9f49dbb4082a5fd35f13fb'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:23.736465 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-9858c2ec-7250-40d5-8b8b-8f52ad053a04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c11e70f0fb9f49dbb4082a5fd35f13fb'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:23.737304 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-9858c2ec-7250-40d5-8b8b-8f52ad053a04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c11e70f0fb9f49dbb4082a5fd35f13fb'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:23.745360 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:20:23.748327 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-9858c2ec-7250-40d5-8b8b-8f52ad053a04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c11e70f0fb9f49dbb4082a5fd35f13fb'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:23.763375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-9858c2ec-7250-40d5-8b8b-8f52ad053a04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c11e70f0fb9f49dbb4082a5fd35f13fb'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:23.779432 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1370/2713] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:20:23 2022] DELETE /identity/v3/users/c11e70f0fb9f49dbb4082a5fd35f13fb => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:20:23.958860 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876483b-1b51-4608-80f5-e74cba525a45 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:20:23.960434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d876483b-1b51-4608-80f5-e74cba525a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:20:23.973966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876483b-1b51-4608-80f5-e74cba525a45 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:20:24.031061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785792d9-8272-4b9e-bfec-5983ee1b5d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:20:24.031816 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785792d9-8272-4b9e-bfec-5983ee1b5d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:20:24.032092 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785792d9-8272-4b9e-bfec-5983ee1b5d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:20:24.032356 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785792d9-8272-4b9e-bfec-5983ee1b5d13 admin admin] PATH_INFO: `/v3/projects/f6fa5023538e423290be27a3a2873b69` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:20:24.033477 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785792d9-8272-4b9e-bfec-5983ee1b5d13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6fa5023538e423290be27a3a2873b69)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:20:24.035398 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785792d9-8272-4b9e-bfec-5983ee1b5d13 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:20:24.085861 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-785792d9-8272-4b9e-bfec-5983ee1b5d13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6fa5023538e423290be27a3a2873b69', 'request_id': 'req-785792d9-8272-4b9e-bfec-5983ee1b5d13'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:24.092592 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-785792d9-8272-4b9e-bfec-5983ee1b5d13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6fa5023538e423290be27a3a2873b69', 'request_id': 'req-785792d9-8272-4b9e-bfec-5983ee1b5d13'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:24.101472 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-785792d9-8272-4b9e-bfec-5983ee1b5d13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6fa5023538e423290be27a3a2873b69', 'request_id': 'req-785792d9-8272-4b9e-bfec-5983ee1b5d13'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:24.113553 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-785792d9-8272-4b9e-bfec-5983ee1b5d13 admin admin] The token cache is being invalidate because project f6fa5023538e423290be27a3a2873b69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:20:24.113846 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-785792d9-8272-4b9e-bfec-5983ee1b5d13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:24.114508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-785792d9-8272-4b9e-bfec-5983ee1b5d13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:20:24.116119 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1344/2714] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:20:23 2022] DELETE /identity/v3/projects/f6fa5023538e423290be27a3a2873b69 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:20:33.775299 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=9,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:20:34.110262 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=71,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:20:44.058339 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858c2ec-7250-40d5-8b8b-8f52ad053a04 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:20:44.060578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-9858c2ec-7250-40d5-8b8b-8f52ad053a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:20:44.144996 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858c2ec-7250-40d5-8b8b-8f52ad053a04 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:20:44.193466 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da4517-c7d6-407d-bb12-81e82df84e66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:20:44.194078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da4517-c7d6-407d-bb12-81e82df84e66 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:20:44.194357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da4517-c7d6-407d-bb12-81e82df84e66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:20:44.194628 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da4517-c7d6-407d-bb12-81e82df84e66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:20:44.196430 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-08da4517-c7d6-407d-bb12-81e82df84e66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:20:44.257228 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da4517-c7d6-407d-bb12-81e82df84e66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:20:44.258662 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da4517-c7d6-407d-bb12-81e82df84e66 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:20:44.268061 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1371/2715] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:20:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:20:51.188659 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2844f-43aa-40af-974a-94d446e7b970 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:20:51.189230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2844f-43aa-40af-974a-94d446e7b970 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:20:51.189230 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2844f-43aa-40af-974a-94d446e7b970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:20:51.210232 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-eaf2844f-43aa-40af-974a-94d446e7b970 None None] MFA Rules not processed for user `ffd12f09606b4e6ba12f63b976df1884`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:20:51.211453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-eaf2844f-43aa-40af-974a-94d446e7b970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:20:51.221215 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1345/2716] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:20:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:20:52.522598 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da4517-c7d6-407d-bb12-81e82df84e66 service cinder] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:20:52.524696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-08da4517-c7d6-407d-bb12-81e82df84e66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:20:52.592559 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:20:52.629187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da4517-c7d6-407d-bb12-81e82df84e66 service cinder] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:20:52.634680 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4d38b-79d5-41fc-903b-01bcfb0fa47a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9b0f54c988e43bda474f978a69fa4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:20:52.635294 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4d38b-79d5-41fc-903b-01bcfb0fa47a service placement] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:20:52.635602 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4d38b-79d5-41fc-903b-01bcfb0fa47a service placement] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:20:52.635875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4d38b-79d5-41fc-903b-01bcfb0fa47a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:20:52.637721 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-dcd4d38b-79d5-41fc-903b-01bcfb0fa47a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:20:52.651076 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4d38b-79d5-41fc-903b-01bcfb0fa47a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:20:52.652745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4d38b-79d5-41fc-903b-01bcfb0fa47a service placement] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:20:52.664192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1372/2717] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:20:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:21:01.205404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:21:02.659396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=18 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:21:30.153803 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6f258-0a39-4803-8994-db3c7bbf1ef8 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:21:30.154272 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6f258-0a39-4803-8994-db3c7bbf1ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:21:30.154394 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6f258-0a39-4803-8994-db3c7bbf1ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:21:30.186996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-8ff6f258-0a39-4803-8994-db3c7bbf1ef8 None None] MFA Rules not processed for user `b3f1ae88e29f4172a7db7393d517a893`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:21:30.188260 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8ff6f258-0a39-4803-8994-db3c7bbf1ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:21:30.254247 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1346/2718] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 23 07:21:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:21:30.304159 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4d38b-79d5-41fc-903b-01bcfb0fa47a service placement] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:21:30.305597 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-dcd4d38b-79d5-41fc-903b-01bcfb0fa47a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:21:30.375950 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4d38b-79d5-41fc-903b-01bcfb0fa47a service placement] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:21:30.381502 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc77ca6-b22e-45eb-b486-f33cc2c739e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd88ec8b477443af95bff9af03471adf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:21:30.382199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc77ca6-b22e-45eb-b486-f33cc2c739e7 service swift] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:21:30.382495 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc77ca6-b22e-45eb-b486-f33cc2c739e7 service swift] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:21:30.382773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc77ca6-b22e-45eb-b486-f33cc2c739e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:21:30.391488 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc77ca6-b22e-45eb-b486-f33cc2c739e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:21:30.393196 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc77ca6-b22e-45eb-b486-f33cc2c739e7 service swift] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:21:30.401881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1373/2719] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 23 07:21:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 07:21:40.247744 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=32 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:21:40.398776 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=32 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:21:43.114616 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd6885-1120-48ac-b24d-12df19661653 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:21:43.115482 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd6885-1120-48ac-b24d-12df19661653 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:21:43.115482 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd6885-1120-48ac-b24d-12df19661653 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:21:43.117114 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1347/2720] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 23 07:21:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:21:43.123735 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c63a8-4c07-49ce-b7b7-9f1e7159a298 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:21:43.124268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c63a8-4c07-49ce-b7b7-9f1e7159a298 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:21:43.124268 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c63a8-4c07-49ce-b7b7-9f1e7159a298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:21:43.167590 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-320c63a8-4c07-49ce-b7b7-9f1e7159a298 None None] MFA Rules not processed for user `a8d2cd93042c4583b63afe629d47fce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:21:43.168823 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-320c63a8-4c07-49ce-b7b7-9f1e7159a298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:21:43.253406 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1374/2721] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 23 07:21:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:21:43.265061 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd6885-1120-48ac-b24d-12df19661653 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:21:43.267049 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-45fd6885-1120-48ac-b24d-12df19661653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:21:43.327370 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd6885-1120-48ac-b24d-12df19661653 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:21:43.331231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b88472-23be-4aba-894b-03e4b4b201d6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57a5f04cb9674b86a6b4b9a472e1afc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:21:43.331875 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b88472-23be-4aba-894b-03e4b4b201d6 service glance] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:21:43.332159 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b88472-23be-4aba-894b-03e4b4b201d6 service glance] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:21:43.332437 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b88472-23be-4aba-894b-03e4b4b201d6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:21:43.343470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b88472-23be-4aba-894b-03e4b4b201d6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:21:43.344961 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b88472-23be-4aba-894b-03e4b4b201d6 service glance] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:21:43.356286 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1348/2722] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:21:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:21:43.430049 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141e40e-e243-479d-843a-6744ee34a0f5 None None] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:21:43.430306 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141e40e-e243-479d-843a-6744ee34a0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:21:43.430582 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141e40e-e243-479d-843a-6744ee34a0f5 None None] PATH_INFO: `/` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:21:43.432033 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1375/2723] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 23 07:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:21:43.436865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de91f6-061f-4d59-a3cd-f676eeaeeedc None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:21:43.437155 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de91f6-061f-4d59-a3cd-f676eeaeeedc None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:21:43.437430 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de91f6-061f-4d59-a3cd-f676eeaeeedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:21:43.475246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-f9de91f6-061f-4d59-a3cd-f676eeaeeedc None None] MFA Rules not processed for user `938364c02d8d4dd2ae832c5129be3cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:21:43.476495 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f9de91f6-061f-4d59-a3cd-f676eeaeeedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:21:43.543586 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1349/2724] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 23 07:21:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:21:43.556305 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141e40e-e243-479d-843a-6744ee34a0f5 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:21:43.558316 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-d141e40e-e243-479d-843a-6744ee34a0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:21:43.573706 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141e40e-e243-479d-843a-6744ee34a0f5 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:21:43.580154 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94b90f-1761-483e-be10-be2a7e62eda5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:21:43.580684 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94b90f-1761-483e-be10-be2a7e62eda5 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:21:43.580967 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94b90f-1761-483e-be10-be2a7e62eda5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:21:43.581289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94b90f-1761-483e-be10-be2a7e62eda5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:21:43.590601 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94b90f-1761-483e-be10-be2a7e62eda5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:21:43.592421 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94b90f-1761-483e-be10-be2a7e62eda5 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:21:43.602458 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1376/2725] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:21:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:21:43.696508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c5291-e888-4a02-bb51-2fcc706794e9 None None] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:21:43.696875 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c5291-e888-4a02-bb51-2fcc706794e9 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:21:43.697241 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c5291-e888-4a02-bb51-2fcc706794e9 None None] PATH_INFO: `/` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:21:43.698692 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1350/2726] 10.222.0.17 () {54 vars in 921 bytes} [Tue Aug 23 07:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:21:43.704672 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3d0bd-73e7-4dc5-a810-20dd13923705 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:21:43.705065 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3d0bd-73e7-4dc5-a810-20dd13923705 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:21:43.705439 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3d0bd-73e7-4dc5-a810-20dd13923705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:21:43.726292 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-34e3d0bd-73e7-4dc5-a810-20dd13923705 None None] MFA Rules not processed for user `a8d2cd93042c4583b63afe629d47fce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:21:43.727575 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-34e3d0bd-73e7-4dc5-a810-20dd13923705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:21:43.737700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1377/2727] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Aug 23 07:21:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:21:43.748821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c5291-e888-4a02-bb51-2fcc706794e9 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:21:43.764356 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c5291-e888-4a02-bb51-2fcc706794e9 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:21:43.769402 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f0d7b-2d99-44c5-aa1a-9c27366a8735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:21:43.770029 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f0d7b-2d99-44c5-aa1a-9c27366a8735 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:21:43.770310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f0d7b-2d99-44c5-aa1a-9c27366a8735 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:21:43.770587 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f0d7b-2d99-44c5-aa1a-9c27366a8735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:21:43.782432 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f0d7b-2d99-44c5-aa1a-9c27366a8735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:21:43.784342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f0d7b-2d99-44c5-aa1a-9c27366a8735 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:21:43.795492 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1351/2728] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:21:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:21:45.427300 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56519f-d833-47f4-99e7-9ace98afe6f7 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:21:45.428082 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56519f-d833-47f4-99e7-9ace98afe6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:21:45.428202 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56519f-d833-47f4-99e7-9ace98afe6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:21:45.449527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-5a56519f-d833-47f4-99e7-9ace98afe6f7 None None] MFA Rules not processed for user `b3f1ae88e29f4172a7db7393d517a893`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:21:45.450775 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-5a56519f-d833-47f4-99e7-9ace98afe6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:21:45.459383 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1378/2729] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Aug 23 07:21:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:21:45.471784 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f0d7b-2d99-44c5-aa1a-9c27366a8735 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:21:45.484049 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f0d7b-2d99-44c5-aa1a-9c27366a8735 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:21:45.489387 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12362846-a493-404b-a21c-ec6b621eaf60 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd88ec8b477443af95bff9af03471adf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:21:45.490035 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12362846-a493-404b-a21c-ec6b621eaf60 service swift] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:21:45.490331 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12362846-a493-404b-a21c-ec6b621eaf60 service swift] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:21:45.490588 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12362846-a493-404b-a21c-ec6b621eaf60 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:21:45.499678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12362846-a493-404b-a21c-ec6b621eaf60 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:21:45.501410 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12362846-a493-404b-a21c-ec6b621eaf60 service swift] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:21:45.511242 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1352/2730] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 23 07:21:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 07:21:55.443886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=56 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:21:55.508097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=70 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:21:59.564046 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eaee37-4744-4b8a-8494-6bcfc3e2d6b8 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:21:59.564572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eaee37-4744-4b8a-8494-6bcfc3e2d6b8 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:21:59.564572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eaee37-4744-4b8a-8494-6bcfc3e2d6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:21:59.585073 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-10eaee37-4744-4b8a-8494-6bcfc3e2d6b8 None None] MFA Rules not processed for user `b3f1ae88e29f4172a7db7393d517a893`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:21:59.586305 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-10eaee37-4744-4b8a-8494-6bcfc3e2d6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:21:59.595070 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1379/2731] 10.222.0.17 () {58 vars in 967 bytes} [Tue Aug 23 07:21:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:21:59.607494 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12362846-a493-404b-a21c-ec6b621eaf60 service swift] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:21:59.618639 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12362846-a493-404b-a21c-ec6b621eaf60 service swift] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:21:59.624118 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab21392-c722-4e84-88fd-c234da0e7c52 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd88ec8b477443af95bff9af03471adf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:21:59.624713 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab21392-c722-4e84-88fd-c234da0e7c52 service swift] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:21:59.624986 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab21392-c722-4e84-88fd-c234da0e7c52 service swift] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:21:59.625270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab21392-c722-4e84-88fd-c234da0e7c52 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:21:59.633478 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab21392-c722-4e84-88fd-c234da0e7c52 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:21:59.635297 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab21392-c722-4e84-88fd-c234da0e7c52 service swift] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:21:59.643076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1353/2732] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Aug 23 07:21:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 07:22:09.579660 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:22:09.639973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=6 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:22:32.490543 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eaee37-4744-4b8a-8494-6bcfc3e2d6b8 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:22:32.500831 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eaee37-4744-4b8a-8494-6bcfc3e2d6b8 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:22:32.505807 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f722b1-0a0f-41fe-ada8-6a5416036c20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:22:32.506412 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f722b1-0a0f-41fe-ada8-6a5416036c20 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:32.506720 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f722b1-0a0f-41fe-ada8-6a5416036c20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:32.506965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f722b1-0a0f-41fe-ada8-6a5416036c20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:32.509226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-57f722b1-0a0f-41fe-ada8-6a5416036c20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:32.526977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f722b1-0a0f-41fe-ada8-6a5416036c20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:22:32.528443 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f722b1-0a0f-41fe-ada8-6a5416036c20 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:22:32.540848 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1380/2733] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:22:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:22:33.750722 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab21392-c722-4e84-88fd-c234da0e7c52 service swift] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:22:33.763574 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab21392-c722-4e84-88fd-c234da0e7c52 service swift] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:22:33.768015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a57ad-b8e4-4290-820b-ea03bd06b078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:22:33.768926 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a57ad-b8e4-4290-820b-ea03bd06b078 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:33.769300 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a57ad-b8e4-4290-820b-ea03bd06b078 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:33.769649 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a57ad-b8e4-4290-820b-ea03bd06b078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:33.772551 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e51a57ad-b8e4-4290-820b-ea03bd06b078 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:33.790240 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a57ad-b8e4-4290-820b-ea03bd06b078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:22:33.791773 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a57ad-b8e4-4290-820b-ea03bd06b078 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:22:33.804155 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1354/2734] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:22:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:22:42.534677 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=6 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:22:42.966692 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f722b1-0a0f-41fe-ada8-6a5416036c20 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:22:42.968844 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-57f722b1-0a0f-41fe-ada8-6a5416036c20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:42.983617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f722b1-0a0f-41fe-ada8-6a5416036c20 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:22:42.990818 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94fe6b0-ae23-471c-a2f2-bb604939d2c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:22:42.991747 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fe6b0-ae23-471c-a2f2-bb604939d2c8 service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:42.992179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fe6b0-ae23-471c-a2f2-bb604939d2c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:42.992662 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fe6b0-ae23-471c-a2f2-bb604939d2c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:42.995618 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c94fe6b0-ae23-471c-a2f2-bb604939d2c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:43.008903 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94fe6b0-ae23-471c-a2f2-bb604939d2c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:22:43.011527 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94fe6b0-ae23-471c-a2f2-bb604939d2c8 service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:22:43.023152 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1381/2735] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:22:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:22:43.092412 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a57ad-b8e4-4290-820b-ea03bd06b078 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:22:43.094843 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e51a57ad-b8e4-4290-820b-ea03bd06b078 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:43.109613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a57ad-b8e4-4290-820b-ea03bd06b078 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:22:43.113986 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863953d4-ff09-4333-820d-5cde5398750b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:22:43.114964 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863953d4-ff09-4333-820d-5cde5398750b service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:43.115751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863953d4-ff09-4333-820d-5cde5398750b service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:43.116475 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863953d4-ff09-4333-820d-5cde5398750b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:43.119164 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-863953d4-ff09-4333-820d-5cde5398750b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:43.134783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863953d4-ff09-4333-820d-5cde5398750b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:22:43.136875 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863953d4-ff09-4333-820d-5cde5398750b service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:22:43.151352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1355/2736] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:22:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:22:43.949648 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94fe6b0-ae23-471c-a2f2-bb604939d2c8 service nova] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:22:43.951110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c94fe6b0-ae23-471c-a2f2-bb604939d2c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:44.026619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94fe6b0-ae23-471c-a2f2-bb604939d2c8 service nova] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:22:44.031620 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71472475-7669-4f02-a0b5-bdce27018596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:22:44.032405 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71472475-7669-4f02-a0b5-bdce27018596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:44.032748 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71472475-7669-4f02-a0b5-bdce27018596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:44.033104 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71472475-7669-4f02-a0b5-bdce27018596 admin admin] PATH_INFO: `/v3/users/4fb28547f4b54c4097d2ffc3c533ca67` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:44.034258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71472475-7669-4f02-a0b5-bdce27018596 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fb28547f4b54c4097d2ffc3c533ca67)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:22:44.036304 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71472475-7669-4f02-a0b5-bdce27018596 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:22:44.089163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-71472475-7669-4f02-a0b5-bdce27018596 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fb28547f4b54c4097d2ffc3c533ca67'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:22:44.090863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-71472475-7669-4f02-a0b5-bdce27018596 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fb28547f4b54c4097d2ffc3c533ca67'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:22:44.091762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-71472475-7669-4f02-a0b5-bdce27018596 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fb28547f4b54c4097d2ffc3c533ca67'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:22:44.104351 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-71472475-7669-4f02-a0b5-bdce27018596 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fb28547f4b54c4097d2ffc3c533ca67'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:22:44.123876 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-71472475-7669-4f02-a0b5-bdce27018596 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fb28547f4b54c4097d2ffc3c533ca67'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:22:44.140553 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1382/2737] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:22:43 2022] DELETE /identity/v3/users/4fb28547f4b54c4097d2ffc3c533ca67 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:22:44.333157 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863953d4-ff09-4333-820d-5cde5398750b service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:22:44.346471 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863953d4-ff09-4333-820d-5cde5398750b service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:22:44.406807 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b3246-dfc2-498d-9236-73b9faef7e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:22:44.407472 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b3246-dfc2-498d-9236-73b9faef7e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:44.407814 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b3246-dfc2-498d-9236-73b9faef7e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:44.408093 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b3246-dfc2-498d-9236-73b9faef7e63 admin admin] PATH_INFO: `/v3/projects/42af08ef367945939b1f59ffa8af351a` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:44.409240 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b3246-dfc2-498d-9236-73b9faef7e63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42af08ef367945939b1f59ffa8af351a)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:22:44.411264 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b3246-dfc2-498d-9236-73b9faef7e63 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:22:44.460831 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f42b3246-dfc2-498d-9236-73b9faef7e63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42af08ef367945939b1f59ffa8af351a', 'request_id': 'req-f42b3246-dfc2-498d-9236-73b9faef7e63'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:22:44.466876 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f42b3246-dfc2-498d-9236-73b9faef7e63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42af08ef367945939b1f59ffa8af351a', 'request_id': 'req-f42b3246-dfc2-498d-9236-73b9faef7e63'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:22:44.475774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f42b3246-dfc2-498d-9236-73b9faef7e63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42af08ef367945939b1f59ffa8af351a', 'request_id': 'req-f42b3246-dfc2-498d-9236-73b9faef7e63'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:22:44.487785 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f42b3246-dfc2-498d-9236-73b9faef7e63 admin admin] The token cache is being invalidate because project 42af08ef367945939b1f59ffa8af351a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:22:44.488076 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f42b3246-dfc2-498d-9236-73b9faef7e63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:22:44.488732 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-f42b3246-dfc2-498d-9236-73b9faef7e63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:22:44.490250 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1356/2738] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:22:44 2022] DELETE /identity/v3/projects/42af08ef367945939b1f59ffa8af351a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:22:44.495497 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799b64b-2732-4beb-813c-84516e912193 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:44.495783 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799b64b-2732-4beb-813c-84516e912193 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:44.496052 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799b64b-2732-4beb-813c-84516e912193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:44.546761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-0799b64b-2732-4beb-813c-84516e912193 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:22:44.548013 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-0799b64b-2732-4beb-813c-84516e912193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:44.591051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:22:44.616399 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1383/2739] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:22:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:22:44.623985 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f78d1-c423-439d-9130-de5d9176b1d4 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:44.624274 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f78d1-c423-439d-9130-de5d9176b1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:44.624543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f78d1-c423-439d-9130-de5d9176b1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:44.778341 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-4a0f78d1-c423-439d-9130-de5d9176b1d4 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:22:44.779603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-4a0f78d1-c423-439d-9130-de5d9176b1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:44.788342 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1357/2740] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:22:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:22:44.792628 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799b64b-2732-4beb-813c-84516e912193 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:22:44.802818 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799b64b-2732-4beb-813c-84516e912193 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:22:44.821194 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73abc8-2477-4180-b86f-0efb8dfe5c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:22:44.822157 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73abc8-2477-4180-b86f-0efb8dfe5c9e admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:44.822645 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73abc8-2477-4180-b86f-0efb8dfe5c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:44.823067 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73abc8-2477-4180-b86f-0efb8dfe5c9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:44.824095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73abc8-2477-4180-b86f-0efb8dfe5c9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:22:44.824482 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73abc8-2477-4180-b86f-0efb8dfe5c9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:22:44.826541 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73abc8-2477-4180-b86f-0efb8dfe5c9e admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:22:44.841507 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1384/2741] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:22:44.846267 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f78d1-c423-439d-9130-de5d9176b1d4 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:22:44.855741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f78d1-c423-439d-9130-de5d9176b1d4 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:22:44.860410 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ca7ca-c11f-4f87-a4c7-5ecffcae3921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:22:44.861014 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ca7ca-c11f-4f87-a4c7-5ecffcae3921 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:44.861338 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ca7ca-c11f-4f87-a4c7-5ecffcae3921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:44.861606 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ca7ca-c11f-4f87-a4c7-5ecffcae3921 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:44.862239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ca7ca-c11f-4f87-a4c7-5ecffcae3921 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:22:44.864198 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ca7ca-c11f-4f87-a4c7-5ecffcae3921 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:22:44.901687 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1358/2742] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:22:44 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:22:44.906021 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73abc8-2477-4180-b86f-0efb8dfe5c9e admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:22:44.918998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73abc8-2477-4180-b86f-0efb8dfe5c9e admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:22:44.982743 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b4be6-dffb-4f93-93e8-93bfddd58524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:22:44.983468 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b4be6-dffb-4f93-93e8-93bfddd58524 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:44.983663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b4be6-dffb-4f93-93e8-93bfddd58524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:44.983936 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b4be6-dffb-4f93-93e8-93bfddd58524 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:44.984574 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b4be6-dffb-4f93-93e8-93bfddd58524 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:22:44.986533 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b4be6-dffb-4f93-93e8-93bfddd58524 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:22:45.013489 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1385/2743] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:22:44 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:22:45.017834 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ca7ca-c11f-4f87-a4c7-5ecffcae3921 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:22:45.031973 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ca7ca-c11f-4f87-a4c7-5ecffcae3921 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:22:45.037042 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223175c-cb66-4b87-8e7a-474bef9cff22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:22:45.037663 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223175c-cb66-4b87-8e7a-474bef9cff22 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:45.037936 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223175c-cb66-4b87-8e7a-474bef9cff22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:45.038203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223175c-cb66-4b87-8e7a-474bef9cff22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:45.038855 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223175c-cb66-4b87-8e7a-474bef9cff22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:22:45.040554 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223175c-cb66-4b87-8e7a-474bef9cff22 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:22:45.054454 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1359/2744] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:22:45.056274 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b4be6-dffb-4f93-93e8-93bfddd58524 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:22:45.072285 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b4be6-dffb-4f93-93e8-93bfddd58524 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:22:45.079511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39af97c-e8d0-4f7f-9fbd-bcebd923706e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:22:45.080358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39af97c-e8d0-4f7f-9fbd-bcebd923706e admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:45.080645 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39af97c-e8d0-4f7f-9fbd-bcebd923706e admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:45.080925 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39af97c-e8d0-4f7f-9fbd-bcebd923706e admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:45.081582 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39af97c-e8d0-4f7f-9fbd-bcebd923706e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:22:45.083333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39af97c-e8d0-4f7f-9fbd-bcebd923706e admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:22:45.096512 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1386/2745] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:22:45.101174 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223175c-cb66-4b87-8e7a-474bef9cff22 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:22:45.113559 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223175c-cb66-4b87-8e7a-474bef9cff22 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:22:45.119466 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c6991-5016-4bd5-81a5-4557db644dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:22:45.120343 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c6991-5016-4bd5-81a5-4557db644dd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:45.120652 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c6991-5016-4bd5-81a5-4557db644dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:45.120933 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c6991-5016-4bd5-81a5-4557db644dd7 admin admin] PATH_INFO: `/v3/projects/cfc6e299d69345a3be46c0052d914742/users/dbf1799461f6485ea88d25d89559906e/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:45.133832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:22:45.137031 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c6991-5016-4bd5-81a5-4557db644dd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfc6e299d69345a3be46c0052d914742, user_id=dbf1799461f6485ea88d25d89559906e, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:22:45.140239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c6991-5016-4bd5-81a5-4557db644dd7 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:22:45.156021 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1360/2746] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:22:45 2022] PUT /identity/v3/projects/cfc6e299d69345a3be46c0052d914742/users/dbf1799461f6485ea88d25d89559906e/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:22:45.162756 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc27e8-de23-478c-9df8-216af8a8c116 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:45.163040 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc27e8-de23-478c-9df8-216af8a8c116 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:45.163314 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc27e8-de23-478c-9df8-216af8a8c116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:45.183152 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-74dc27e8-de23-478c-9df8-216af8a8c116 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:22:45.184398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-74dc27e8-de23-478c-9df8-216af8a8c116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:45.253014 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1387/2747] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:22:45.263363 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c6991-5016-4bd5-81a5-4557db644dd7 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:22:45.265307 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3a2c6991-5016-4bd5-81a5-4557db644dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:45.325848 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c6991-5016-4bd5-81a5-4557db644dd7 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:22:45.330612 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cddb1-9e7d-4acc-8cbb-e83a9ebdb0f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:22:45.331602 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cddb1-9e7d-4acc-8cbb-e83a9ebdb0f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:45.331957 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cddb1-9e7d-4acc-8cbb-e83a9ebdb0f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:45.332213 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cddb1-9e7d-4acc-8cbb-e83a9ebdb0f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:45.344176 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cddb1-9e7d-4acc-8cbb-e83a9ebdb0f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:22:45.345712 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cddb1-9e7d-4acc-8cbb-e83a9ebdb0f3 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:22:45.356540 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1361/2748] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:22:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:22:49.439729 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b0733-a6c2-46df-8f83-6b52ef3ac59a None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:49.440234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b0733-a6c2-46df-8f83-6b52ef3ac59a None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:49.440323 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b0733-a6c2-46df-8f83-6b52ef3ac59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:49.483131 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-3f3b0733-a6c2-46df-8f83-6b52ef3ac59a None None] MFA Rules not processed for user `dbf1799461f6485ea88d25d89559906e`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:22:49.486300 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3f3b0733-a6c2-46df-8f83-6b52ef3ac59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:49.535824 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:22:49.569732 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1388/2749] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:22:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:22:49.585075 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cddb1-9e7d-4acc-8cbb-e83a9ebdb0f3 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:22:49.586713 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-a65cddb1-9e7d-4acc-8cbb-e83a9ebdb0f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:49.691825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cddb1-9e7d-4acc-8cbb-e83a9ebdb0f3 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:22:49.698066 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4000f6-5186-4297-8b7e-4bf7d8970cbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:22:49.698404 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4000f6-5186-4297-8b7e-4bf7d8970cbe service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:49.698682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4000f6-5186-4297-8b7e-4bf7d8970cbe service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:49.698948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4000f6-5186-4297-8b7e-4bf7d8970cbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:49.707565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4000f6-5186-4297-8b7e-4bf7d8970cbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:22:49.709306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4000f6-5186-4297-8b7e-4bf7d8970cbe service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:22:49.721051 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1362/2750] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:22:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:22:51.387760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d2202-c353-422f-b67a-8db720e78331 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:51.388275 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d2202-c353-422f-b67a-8db720e78331 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:51.388275 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d2202-c353-422f-b67a-8db720e78331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:51.410569 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-b24d2202-c353-422f-b67a-8db720e78331 None None] MFA Rules not processed for user `dbf1799461f6485ea88d25d89559906e`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:22:51.411880 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b24d2202-c353-422f-b67a-8db720e78331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:51.419331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1389/2751] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:22:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:22:52.547220 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4000f6-5186-4297-8b7e-4bf7d8970cbe service nova] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:22:52.548819 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-cc4000f6-5186-4297-8b7e-4bf7d8970cbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:52.638438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4000f6-5186-4297-8b7e-4bf7d8970cbe service nova] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:22:52.645643 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a4bfc-7c66-4871-ab75-ecf129491244 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9b0f54c988e43bda474f978a69fa4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:22:52.646521 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a4bfc-7c66-4871-ab75-ecf129491244 service placement] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:52.647009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a4bfc-7c66-4871-ab75-ecf129491244 service placement] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:52.647543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a4bfc-7c66-4871-ab75-ecf129491244 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:52.649900 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-397a4bfc-7c66-4871-ab75-ecf129491244 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:52.666771 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a4bfc-7c66-4871-ab75-ecf129491244 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:22:52.668769 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a4bfc-7c66-4871-ab75-ecf129491244 service placement] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:22:52.688128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1363/2752] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:22:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:22:56.307901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d2202-c353-422f-b67a-8db720e78331 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:22:56.318858 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d2202-c353-422f-b67a-8db720e78331 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:22:56.324108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac2217-2a58-49c9-baa5-c8a248841b91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:22:56.325169 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac2217-2a58-49c9-baa5-c8a248841b91 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:56.325575 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac2217-2a58-49c9-baa5-c8a248841b91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:56.325973 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac2217-2a58-49c9-baa5-c8a248841b91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:56.329094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-35ac2217-2a58-49c9-baa5-c8a248841b91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:56.346034 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac2217-2a58-49c9-baa5-c8a248841b91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:22:56.347531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac2217-2a58-49c9-baa5-c8a248841b91 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:22:56.358255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1390/2753] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:22:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:22:59.902043 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a4bfc-7c66-4871-ab75-ecf129491244 service placement] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:22:59.906145 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-397a4bfc-7c66-4871-ab75-ecf129491244 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:59.926524 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a4bfc-7c66-4871-ab75-ecf129491244 service placement] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:22:59.935393 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551284d-3748-4f6b-81ea-63117cecddf3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9b0f54c988e43bda474f978a69fa4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:22:59.936045 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551284d-3748-4f6b-81ea-63117cecddf3 service placement] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:22:59.936319 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551284d-3748-4f6b-81ea-63117cecddf3 service placement] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:22:59.936588 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551284d-3748-4f6b-81ea-63117cecddf3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:22:59.939392 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e551284d-3748-4f6b-81ea-63117cecddf3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:22:59.952346 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:22:59.956434 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e551284d-3748-4f6b-81ea-63117cecddf3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:22:59.962535 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e551284d-3748-4f6b-81ea-63117cecddf3 service placement] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:22:59.976169 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1364/2754] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:22:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:23:03.435127 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac2217-2a58-49c9-baa5-c8a248841b91 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:23:03.437316 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-35ac2217-2a58-49c9-baa5-c8a248841b91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:23:03.451251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac2217-2a58-49c9-baa5-c8a248841b91 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:23:03.455986 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bea12e-771b-400e-8046-edea42e2bbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:23:03.456670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bea12e-771b-400e-8046-edea42e2bbcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:23:03.457105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bea12e-771b-400e-8046-edea42e2bbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:23:03.457590 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bea12e-771b-400e-8046-edea42e2bbcb admin admin] PATH_INFO: `/v3/users/ffd12f09606b4e6ba12f63b976df1884` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:23:03.458783 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bea12e-771b-400e-8046-edea42e2bbcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffd12f09606b4e6ba12f63b976df1884)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:23:03.460810 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bea12e-771b-400e-8046-edea42e2bbcb admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:23:03.515578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f4bea12e-771b-400e-8046-edea42e2bbcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffd12f09606b4e6ba12f63b976df1884'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:03.516782 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f4bea12e-771b-400e-8046-edea42e2bbcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffd12f09606b4e6ba12f63b976df1884'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:03.517412 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f4bea12e-771b-400e-8046-edea42e2bbcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffd12f09606b4e6ba12f63b976df1884'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:03.526497 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f4bea12e-771b-400e-8046-edea42e2bbcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffd12f09606b4e6ba12f63b976df1884'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:03.540313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-f4bea12e-771b-400e-8046-edea42e2bbcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffd12f09606b4e6ba12f63b976df1884'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:03.554957 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1391/2755] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:23:03 2022] DELETE /identity/v3/users/ffd12f09606b4e6ba12f63b976df1884 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:23:03.710565 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551284d-3748-4f6b-81ea-63117cecddf3 service placement] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:23:03.712130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e551284d-3748-4f6b-81ea-63117cecddf3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:23:03.730887 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551284d-3748-4f6b-81ea-63117cecddf3 service placement] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:23:03.768203 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bea12e-771b-400e-8046-edea42e2bbcb admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:23:03.770070 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-f4bea12e-771b-400e-8046-edea42e2bbcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:23:03.839451 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca95cd-b4c3-4eaa-8023-b81b4bc9dcdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:23:03.840074 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca95cd-b4c3-4eaa-8023-b81b4bc9dcdf service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:23:03.840354 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca95cd-b4c3-4eaa-8023-b81b4bc9dcdf service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:23:03.840628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca95cd-b4c3-4eaa-8023-b81b4bc9dcdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:23:03.842409 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-42ca95cd-b4c3-4eaa-8023-b81b4bc9dcdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:23:03.851809 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bea12e-771b-400e-8046-edea42e2bbcb admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:23:03.857208 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8793d-ed91-4ff4-83b3-3ddef98364ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:23:03.857831 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca95cd-b4c3-4eaa-8023-b81b4bc9dcdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:23:03.857968 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8793d-ed91-4ff4-83b3-3ddef98364ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:23:03.858345 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8793d-ed91-4ff4-83b3-3ddef98364ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:23:03.858687 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8793d-ed91-4ff4-83b3-3ddef98364ed admin admin] PATH_INFO: `/v3/projects/f43b271d00f74eb7aa3d0719315c5a99` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:23:03.859627 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca95cd-b4c3-4eaa-8023-b81b4bc9dcdf service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:23:03.860017 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8793d-ed91-4ff4-83b3-3ddef98364ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=f43b271d00f74eb7aa3d0719315c5a99)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:23:03.862013 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8793d-ed91-4ff4-83b3-3ddef98364ed admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:23:03.872774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1365/2756] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:23:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:23:03.909929 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:23:03.931249 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-47b8793d-ed91-4ff4-83b3-3ddef98364ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f43b271d00f74eb7aa3d0719315c5a99', 'request_id': 'req-47b8793d-ed91-4ff4-83b3-3ddef98364ed'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:03.937534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-47b8793d-ed91-4ff4-83b3-3ddef98364ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f43b271d00f74eb7aa3d0719315c5a99', 'request_id': 'req-47b8793d-ed91-4ff4-83b3-3ddef98364ed'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:03.946173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-47b8793d-ed91-4ff4-83b3-3ddef98364ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f43b271d00f74eb7aa3d0719315c5a99', 'request_id': 'req-47b8793d-ed91-4ff4-83b3-3ddef98364ed'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:03.968034 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-47b8793d-ed91-4ff4-83b3-3ddef98364ed admin admin] The token cache is being invalidate because project f43b271d00f74eb7aa3d0719315c5a99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:23:03.968647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-47b8793d-ed91-4ff4-83b3-3ddef98364ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:03.969969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-47b8793d-ed91-4ff4-83b3-3ddef98364ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:03.971775 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1392/2757] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:23:03 2022] DELETE /identity/v3/projects/f43b271d00f74eb7aa3d0719315c5a99 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:23:03.975922 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca95cd-b4c3-4eaa-8023-b81b4bc9dcdf service nova] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:23:03.989091 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca95cd-b4c3-4eaa-8023-b81b4bc9dcdf service nova] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:23:04.062539 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841901e-f641-441c-8c9c-97d317b62065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:23:04.063187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841901e-f641-441c-8c9c-97d317b62065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:23:04.063568 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841901e-f641-441c-8c9c-97d317b62065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:23:04.064470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841901e-f641-441c-8c9c-97d317b62065 admin admin] PATH_INFO: `/v3/users/6a9b12585539482c945df41a8b6b6420` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:23:04.065570 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841901e-f641-441c-8c9c-97d317b62065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a9b12585539482c945df41a8b6b6420)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:23:04.067537 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841901e-f641-441c-8c9c-97d317b62065 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:23:04.124423 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-8841901e-f641-441c-8c9c-97d317b62065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a9b12585539482c945df41a8b6b6420'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:04.125232 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-8841901e-f641-441c-8c9c-97d317b62065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a9b12585539482c945df41a8b6b6420'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:04.125865 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-8841901e-f641-441c-8c9c-97d317b62065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a9b12585539482c945df41a8b6b6420'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:04.133532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-8841901e-f641-441c-8c9c-97d317b62065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a9b12585539482c945df41a8b6b6420'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:04.151125 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-8841901e-f641-441c-8c9c-97d317b62065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a9b12585539482c945df41a8b6b6420'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:04.163978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:23:04.170635 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1366/2758] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:23:03 2022] DELETE /identity/v3/users/6a9b12585539482c945df41a8b6b6420 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:23:04.362313 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8793d-ed91-4ff4-83b3-3ddef98364ed admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:23:04.363970 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-47b8793d-ed91-4ff4-83b3-3ddef98364ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:23:04.380514 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8793d-ed91-4ff4-83b3-3ddef98364ed admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:23:04.443953 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d68b24-c6bc-48ba-b3e1-89d5736334fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:23:04.444773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d68b24-c6bc-48ba-b3e1-89d5736334fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:23:04.445171 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d68b24-c6bc-48ba-b3e1-89d5736334fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:23:04.445455 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d68b24-c6bc-48ba-b3e1-89d5736334fb admin admin] PATH_INFO: `/v3/projects/5d08e1b2b59d46499716a4c4d55e2b0c` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:23:04.447871 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d68b24-c6bc-48ba-b3e1-89d5736334fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d08e1b2b59d46499716a4c4d55e2b0c)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:23:04.449892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d68b24-c6bc-48ba-b3e1-89d5736334fb admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:23:04.510469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-17d68b24-c6bc-48ba-b3e1-89d5736334fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d08e1b2b59d46499716a4c4d55e2b0c', 'request_id': 'req-17d68b24-c6bc-48ba-b3e1-89d5736334fb'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:04.518822 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-17d68b24-c6bc-48ba-b3e1-89d5736334fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d08e1b2b59d46499716a4c4d55e2b0c', 'request_id': 'req-17d68b24-c6bc-48ba-b3e1-89d5736334fb'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:04.526878 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-17d68b24-c6bc-48ba-b3e1-89d5736334fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d08e1b2b59d46499716a4c4d55e2b0c', 'request_id': 'req-17d68b24-c6bc-48ba-b3e1-89d5736334fb'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:04.538164 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-17d68b24-c6bc-48ba-b3e1-89d5736334fb admin admin] The token cache is being invalidate because project 5d08e1b2b59d46499716a4c4d55e2b0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:23:04.538478 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-17d68b24-c6bc-48ba-b3e1-89d5736334fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:04.539108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-17d68b24-c6bc-48ba-b3e1-89d5736334fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:23:04.540605 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1393/2759] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:23:04 2022] DELETE /identity/v3/projects/5d08e1b2b59d46499716a4c4d55e2b0c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:23:14.535406 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:DELETE=12,keystone:SELECT=62,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:25:35.504296 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaddb7b-8c12-4457-8e09-2911fc65175e None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:25:35.504858 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaddb7b-8c12-4457-8e09-2911fc65175e None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:25:35.504858 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaddb7b-8c12-4457-8e09-2911fc65175e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:25:35.525387 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-edaddb7b-8c12-4457-8e09-2911fc65175e None None] MFA Rules not processed for user `dbf1799461f6485ea88d25d89559906e`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:25:35.526584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-edaddb7b-8c12-4457-8e09-2911fc65175e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:25:35.589228 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1367/2760] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:25:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:25:35.915800 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d68b24-c6bc-48ba-b3e1-89d5736334fb admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:25:35.917282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-17d68b24-c6bc-48ba-b3e1-89d5736334fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:25:35.984516 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d68b24-c6bc-48ba-b3e1-89d5736334fb admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:25:35.988741 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc37b8-830d-427b-8994-85c8315da109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:25:35.989345 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc37b8-830d-427b-8994-85c8315da109 service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:25:35.989626 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc37b8-830d-427b-8994-85c8315da109 service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:25:35.989894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc37b8-830d-427b-8994-85c8315da109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:25:35.992063 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a6cc37b8-830d-427b-8994-85c8315da109 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:25:36.002998 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc37b8-830d-427b-8994-85c8315da109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:25:36.004461 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc37b8-830d-427b-8994-85c8315da109 service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:25:36.012745 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1394/2761] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:25:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:25:44.879550 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaddb7b-8c12-4457-8e09-2911fc65175e None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:25:44.881102 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-edaddb7b-8c12-4457-8e09-2911fc65175e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:25:44.989288 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaddb7b-8c12-4457-8e09-2911fc65175e None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:25:44.996189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17c093-1b2c-4471-8aec-40ef827d05e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:25:44.997017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17c093-1b2c-4471-8aec-40ef827d05e1 service nova] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:25:44.997352 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17c093-1b2c-4471-8aec-40ef827d05e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:25:44.997668 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17c093-1b2c-4471-8aec-40ef827d05e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:25:44.999686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-af17c093-1b2c-4471-8aec-40ef827d05e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:25:45.014218 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17c093-1b2c-4471-8aec-40ef827d05e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:25:45.016066 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17c093-1b2c-4471-8aec-40ef827d05e1 service nova] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:25:45.028870 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1368/2762] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:25:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:25:46.009181 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=26 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:25:55.023094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=59 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:26:36.080714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc37b8-830d-427b-8994-85c8315da109 service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:36.083048 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-a6cc37b8-830d-427b-8994-85c8315da109 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:26:36.158056 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc37b8-830d-427b-8994-85c8315da109 service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:36.163300 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271f3cf-e3ce-46ae-aab2-9b55b48295d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9b0f54c988e43bda474f978a69fa4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:36.163994 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271f3cf-e3ce-46ae-aab2-9b55b48295d0 service placement] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:36.164271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271f3cf-e3ce-46ae-aab2-9b55b48295d0 service placement] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:36.164540 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271f3cf-e3ce-46ae-aab2-9b55b48295d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:36.166299 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b271f3cf-e3ce-46ae-aab2-9b55b48295d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:26:36.178873 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271f3cf-e3ce-46ae-aab2-9b55b48295d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:36.180645 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271f3cf-e3ce-46ae-aab2-9b55b48295d0 service placement] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:36.193726 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1395/2763] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:26:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:26:46.186484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=32 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:26:55.135601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17c093-1b2c-4471-8aec-40ef827d05e1 service nova] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:55.147882 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17c093-1b2c-4471-8aec-40ef827d05e1 service nova] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:55.154000 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93bce3-ba01-4212-bb94-0086b7671746 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9b0f54c988e43bda474f978a69fa4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:55.154613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93bce3-ba01-4212-bb94-0086b7671746 service placement] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:55.154897 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93bce3-ba01-4212-bb94-0086b7671746 service placement] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:55.155173 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93bce3-ba01-4212-bb94-0086b7671746 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:55.157126 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-dd93bce3-ba01-4212-bb94-0086b7671746 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:26:55.170335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93bce3-ba01-4212-bb94-0086b7671746 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:55.171571 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93bce3-ba01-4212-bb94-0086b7671746 service placement] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:55.193049 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1369/2764] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:26:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:26:55.193566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271f3cf-e3ce-46ae-aab2-9b55b48295d0 service placement] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:55.195282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b271f3cf-e3ce-46ae-aab2-9b55b48295d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:26:55.267079 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271f3cf-e3ce-46ae-aab2-9b55b48295d0 service placement] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:55.272415 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f233184-aedd-4416-8a49-bb93fd3f6d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:55.273099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f233184-aedd-4416-8a49-bb93fd3f6d0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:55.273374 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f233184-aedd-4416-8a49-bb93fd3f6d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:55.273645 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f233184-aedd-4416-8a49-bb93fd3f6d0f admin admin] PATH_INFO: `/v3/users/dbf1799461f6485ea88d25d89559906e` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:55.274852 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f233184-aedd-4416-8a49-bb93fd3f6d0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbf1799461f6485ea88d25d89559906e)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:55.276862 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f233184-aedd-4416-8a49-bb93fd3f6d0f admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:55.327837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1f233184-aedd-4416-8a49-bb93fd3f6d0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbf1799461f6485ea88d25d89559906e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:55.328464 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1f233184-aedd-4416-8a49-bb93fd3f6d0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbf1799461f6485ea88d25d89559906e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:55.329077 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1f233184-aedd-4416-8a49-bb93fd3f6d0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbf1799461f6485ea88d25d89559906e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:55.336349 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1f233184-aedd-4416-8a49-bb93fd3f6d0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbf1799461f6485ea88d25d89559906e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:55.348716 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-1f233184-aedd-4416-8a49-bb93fd3f6d0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbf1799461f6485ea88d25d89559906e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:55.362858 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1396/2765] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:26:55 2022] DELETE /identity/v3/users/dbf1799461f6485ea88d25d89559906e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:26:55.541967 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93bce3-ba01-4212-bb94-0086b7671746 service placement] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:55.553174 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93bce3-ba01-4212-bb94-0086b7671746 service placement] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:55.612978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9e183-6200-4a41-9737-13928859a8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:55.613613 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9e183-6200-4a41-9737-13928859a8bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:55.613892 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9e183-6200-4a41-9737-13928859a8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:55.614165 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9e183-6200-4a41-9737-13928859a8bc admin admin] PATH_INFO: `/v3/projects/cfc6e299d69345a3be46c0052d914742` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:55.615355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9e183-6200-4a41-9737-13928859a8bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfc6e299d69345a3be46c0052d914742)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:55.617353 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9e183-6200-4a41-9737-13928859a8bc admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:55.665951 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-98c9e183-6200-4a41-9737-13928859a8bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfc6e299d69345a3be46c0052d914742', 'request_id': 'req-98c9e183-6200-4a41-9737-13928859a8bc'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:55.671991 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-98c9e183-6200-4a41-9737-13928859a8bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfc6e299d69345a3be46c0052d914742', 'request_id': 'req-98c9e183-6200-4a41-9737-13928859a8bc'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:55.681782 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-98c9e183-6200-4a41-9737-13928859a8bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfc6e299d69345a3be46c0052d914742', 'request_id': 'req-98c9e183-6200-4a41-9737-13928859a8bc'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:55.693949 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-98c9e183-6200-4a41-9737-13928859a8bc admin admin] The token cache is being invalidate because project cfc6e299d69345a3be46c0052d914742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:26:55.694235 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-98c9e183-6200-4a41-9737-13928859a8bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:55.694931 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-98c9e183-6200-4a41-9737-13928859a8bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:55.696544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1370/2766] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:26:55 2022] DELETE /identity/v3/projects/cfc6e299d69345a3be46c0052d914742 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:26:55.701467 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef8582-0441-48c7-8c14-f505dea000af None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:55.701752 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef8582-0441-48c7-8c14-f505dea000af None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:55.702024 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef8582-0441-48c7-8c14-f505dea000af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:55.722595 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-f7ef8582-0441-48c7-8c14-f505dea000af None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:26:55.724062 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-f7ef8582-0441-48c7-8c14-f505dea000af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:26:55.787843 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1397/2767] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:26:55.795753 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b72b0-8c22-4390-aefe-7ab127480c58 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:55.796038 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b72b0-8c22-4390-aefe-7ab127480c58 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:55.796295 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b72b0-8c22-4390-aefe-7ab127480c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:55.815903 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-683b72b0-8c22-4390-aefe-7ab127480c58 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:26:55.817109 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-683b72b0-8c22-4390-aefe-7ab127480c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:26:55.828964 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1371/2768] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:26:55.833969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef8582-0441-48c7-8c14-f505dea000af None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:55.845282 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef8582-0441-48c7-8c14-f505dea000af None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:55.855891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3850b37-e08b-47bf-b67b-d1706dd269be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:55.855891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3850b37-e08b-47bf-b67b-d1706dd269be admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:55.855891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3850b37-e08b-47bf-b67b-d1706dd269be admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:55.855891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3850b37-e08b-47bf-b67b-d1706dd269be admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:55.856384 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3850b37-e08b-47bf-b67b-d1706dd269be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:26:55.856384 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3850b37-e08b-47bf-b67b-d1706dd269be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:55.856384 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3850b37-e08b-47bf-b67b-d1706dd269be admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:55.867403 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1398/2769] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:26:55.871847 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b72b0-8c22-4390-aefe-7ab127480c58 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:55.881580 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b72b0-8c22-4390-aefe-7ab127480c58 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:55.886453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0a4c8-85ad-4fde-a6e1-35f3759033b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:55.887056 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0a4c8-85ad-4fde-a6e1-35f3759033b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:55.887334 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0a4c8-85ad-4fde-a6e1-35f3759033b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:55.887631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0a4c8-85ad-4fde-a6e1-35f3759033b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:55.888254 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0a4c8-85ad-4fde-a6e1-35f3759033b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:55.890184 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0a4c8-85ad-4fde-a6e1-35f3759033b0 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:55.917943 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1372/2770] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:26:55 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:26:55.922254 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3850b37-e08b-47bf-b67b-d1706dd269be admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:55.932905 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3850b37-e08b-47bf-b67b-d1706dd269be admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:55.937092 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:26:56.002634 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94462400-1403-4394-9b75-52accbb2c395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:56.003365 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94462400-1403-4394-9b75-52accbb2c395 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:56.003796 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94462400-1403-4394-9b75-52accbb2c395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:56.004284 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94462400-1403-4394-9b75-52accbb2c395 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:56.005213 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94462400-1403-4394-9b75-52accbb2c395 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:56.008229 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94462400-1403-4394-9b75-52accbb2c395 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:56.038863 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1399/2771] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:26:55 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:26:56.045503 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0a4c8-85ad-4fde-a6e1-35f3759033b0 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:56.057233 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0a4c8-85ad-4fde-a6e1-35f3759033b0 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:56.062900 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e72e95-1ee6-4bad-9dcd-94789b36a4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:56.063572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e72e95-1ee6-4bad-9dcd-94789b36a4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:56.064089 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e72e95-1ee6-4bad-9dcd-94789b36a4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:56.064196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e72e95-1ee6-4bad-9dcd-94789b36a4a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:56.064761 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e72e95-1ee6-4bad-9dcd-94789b36a4a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:56.066435 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e72e95-1ee6-4bad-9dcd-94789b36a4a4 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:56.076738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1373/2772] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:26:56.081638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94462400-1403-4394-9b75-52accbb2c395 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:56.097491 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94462400-1403-4394-9b75-52accbb2c395 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:56.105020 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72dc617-5c41-4a1b-b4f5-1c08aaa92ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:56.105619 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dc617-5c41-4a1b-b4f5-1c08aaa92ece admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:56.105892 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dc617-5c41-4a1b-b4f5-1c08aaa92ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:56.106159 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dc617-5c41-4a1b-b4f5-1c08aaa92ece admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:56.106790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72dc617-5c41-4a1b-b4f5-1c08aaa92ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:56.108442 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72dc617-5c41-4a1b-b4f5-1c08aaa92ece admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:56.120425 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1400/2773] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:26:56.124750 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e72e95-1ee6-4bad-9dcd-94789b36a4a4 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:56.134785 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e72e95-1ee6-4bad-9dcd-94789b36a4a4 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:56.139682 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7899434-5f30-4d60-b3a0-b16444bbd53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:56.140318 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7899434-5f30-4d60-b3a0-b16444bbd53a admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:56.140590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7899434-5f30-4d60-b3a0-b16444bbd53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:56.140867 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7899434-5f30-4d60-b3a0-b16444bbd53a admin admin] PATH_INFO: `/v3/projects/f744a0f1eb624db1aeb75dab7a81323b/users/b7df6740c4254f4692fe72d60c9dcaf5/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:56.157221 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7899434-5f30-4d60-b3a0-b16444bbd53a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f744a0f1eb624db1aeb75dab7a81323b, user_id=b7df6740c4254f4692fe72d60c9dcaf5, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:56.160418 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7899434-5f30-4d60-b3a0-b16444bbd53a admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:56.177399 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1374/2774] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:26:56 2022] PUT /identity/v3/projects/f744a0f1eb624db1aeb75dab7a81323b/users/b7df6740c4254f4692fe72d60c9dcaf5/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:26:56.241083 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4f85f-38b7-4be4-9029-ded4c8cbcb11 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:56.241766 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4f85f-38b7-4be4-9029-ded4c8cbcb11 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:56.241766 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4f85f-38b7-4be4-9029-ded4c8cbcb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:56.275004 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-00b4f85f-38b7-4be4-9029-ded4c8cbcb11 None None] MFA Rules not processed for user `b7df6740c4254f4692fe72d60c9dcaf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:26:56.276164 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-00b4f85f-38b7-4be4-9029-ded4c8cbcb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:26:56.346226 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1401/2775] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:26:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:26:56.353190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7899434-5f30-4d60-b3a0-b16444bbd53a admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:56.375142 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:26:56.427446 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7899434-5f30-4d60-b3a0-b16444bbd53a admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:56.432710 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b600fb-8fef-4017-b20f-a21fa15bdd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:56.433355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b600fb-8fef-4017-b20f-a21fa15bdd92 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:56.433633 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b600fb-8fef-4017-b20f-a21fa15bdd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:56.433901 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b600fb-8fef-4017-b20f-a21fa15bdd92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:56.434553 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b600fb-8fef-4017-b20f-a21fa15bdd92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:56.436511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b600fb-8fef-4017-b20f-a21fa15bdd92 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:56.470090 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1375/2776] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:26:56 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:26:56.471756 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4f85f-38b7-4be4-9029-ded4c8cbcb11 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:56.483680 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4f85f-38b7-4be4-9029-ded4c8cbcb11 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:56.542768 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ba15e-08ae-4ea1-9633-a383fb8242ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:56.543300 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ba15e-08ae-4ea1-9633-a383fb8242ba admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:56.543612 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ba15e-08ae-4ea1-9633-a383fb8242ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:56.543870 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ba15e-08ae-4ea1-9633-a383fb8242ba admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:56.544496 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ba15e-08ae-4ea1-9633-a383fb8242ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:56.546413 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ba15e-08ae-4ea1-9633-a383fb8242ba admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:56.570704 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1402/2777] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:26:56 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:26:56.574934 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b600fb-8fef-4017-b20f-a21fa15bdd92 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:56.586780 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b600fb-8fef-4017-b20f-a21fa15bdd92 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:56.591856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383c705-0853-4ddf-963d-dc17caeb330f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:56.592462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383c705-0853-4ddf-963d-dc17caeb330f admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:56.592739 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383c705-0853-4ddf-963d-dc17caeb330f admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:56.593007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383c705-0853-4ddf-963d-dc17caeb330f admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:56.593650 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5383c705-0853-4ddf-963d-dc17caeb330f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:56.595977 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5383c705-0853-4ddf-963d-dc17caeb330f admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:56.607546 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1376/2778] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:26:56.611970 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ba15e-08ae-4ea1-9633-a383fb8242ba admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:56.618692 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:26:56.623993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ba15e-08ae-4ea1-9633-a383fb8242ba admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:56.630836 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95507713-c0f0-4a05-afd7-7ec913162e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:56.631538 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95507713-c0f0-4a05-afd7-7ec913162e4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:56.631820 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95507713-c0f0-4a05-afd7-7ec913162e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:56.632095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95507713-c0f0-4a05-afd7-7ec913162e4e admin admin] PATH_INFO: `/v3/projects/3545782601024658a00a6904f7e22d1a/users/28e1f66e0d094cd1a23d73f38c0852ff/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:56.647749 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95507713-c0f0-4a05-afd7-7ec913162e4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3545782601024658a00a6904f7e22d1a, user_id=28e1f66e0d094cd1a23d73f38c0852ff, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:56.650445 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95507713-c0f0-4a05-afd7-7ec913162e4e admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:56.665702 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1403/2779] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:26:56 2022] PUT /identity/v3/projects/3545782601024658a00a6904f7e22d1a/users/28e1f66e0d094cd1a23d73f38c0852ff/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:26:56.669830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383c705-0853-4ddf-963d-dc17caeb330f admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:56.680491 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383c705-0853-4ddf-963d-dc17caeb330f admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:56.738968 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ecbc31-9820-465f-b7e1-4a99c9ddf400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:56.739611 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecbc31-9820-465f-b7e1-4a99c9ddf400 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:56.739900 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecbc31-9820-465f-b7e1-4a99c9ddf400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:56.740152 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecbc31-9820-465f-b7e1-4a99c9ddf400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:56.740786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ecbc31-9820-465f-b7e1-4a99c9ddf400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:56.742397 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ecbc31-9820-465f-b7e1-4a99c9ddf400 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:56.752123 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1377/2780] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:26:56.756399 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95507713-c0f0-4a05-afd7-7ec913162e4e admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:56.766866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95507713-c0f0-4a05-afd7-7ec913162e4e admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:56.771782 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd773e-6b65-4128-b37a-7d0dca545d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:56.772419 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd773e-6b65-4128-b37a-7d0dca545d29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:56.772699 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd773e-6b65-4128-b37a-7d0dca545d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:56.772969 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd773e-6b65-4128-b37a-7d0dca545d29 admin admin] PATH_INFO: `/v3/projects/3545782601024658a00a6904f7e22d1a/users/28e1f66e0d094cd1a23d73f38c0852ff/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:56.774888 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd773e-6b65-4128-b37a-7d0dca545d29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3545782601024658a00a6904f7e22d1a, user_id=28e1f66e0d094cd1a23d73f38c0852ff, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:56.777544 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd773e-6b65-4128-b37a-7d0dca545d29 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:56.787625 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1404/2781] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:26:56 2022] PUT /identity/v3/projects/3545782601024658a00a6904f7e22d1a/users/28e1f66e0d094cd1a23d73f38c0852ff/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:26:56.797388 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ee6f5-f6ee-483c-9286-ade8e32d21cb None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:56.797610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ee6f5-f6ee-483c-9286-ade8e32d21cb None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:56.797938 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ee6f5-f6ee-483c-9286-ade8e32d21cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:56.830493 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-941ee6f5-f6ee-483c-9286-ade8e32d21cb None None] MFA Rules not processed for user `28e1f66e0d094cd1a23d73f38c0852ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:26:56.831868 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-941ee6f5-f6ee-483c-9286-ade8e32d21cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:26:56.899104 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1378/2782] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:26:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:26:56.911107 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd773e-6b65-4128-b37a-7d0dca545d29 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:56.913616 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-98bd773e-6b65-4128-b37a-7d0dca545d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:26:56.975833 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd773e-6b65-4128-b37a-7d0dca545d29 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:56.979828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bee0e-2282-40da-99cf-7278949a8f04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:56.980444 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bee0e-2282-40da-99cf-7278949a8f04 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:56.980716 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bee0e-2282-40da-99cf-7278949a8f04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:56.980985 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bee0e-2282-40da-99cf-7278949a8f04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:56.990758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626bee0e-2282-40da-99cf-7278949a8f04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:56.992223 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626bee0e-2282-40da-99cf-7278949a8f04 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:57.002121 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1405/2783] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:26:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:26:57.419349 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ee6f5-f6ee-483c-9286-ade8e32d21cb None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:57.429384 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:26:57.484880 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ee6f5-f6ee-483c-9286-ade8e32d21cb None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:57.489948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15416f-92dc-4d64-a8b5-c063ca053e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:57.490564 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15416f-92dc-4d64-a8b5-c063ca053e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:57.490840 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15416f-92dc-4d64-a8b5-c063ca053e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:57.491138 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15416f-92dc-4d64-a8b5-c063ca053e6e admin admin] PATH_INFO: `/v3/users/b7df6740c4254f4692fe72d60c9dcaf5` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:57.492280 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15416f-92dc-4d64-a8b5-c063ca053e6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7df6740c4254f4692fe72d60c9dcaf5)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:57.494233 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15416f-92dc-4d64-a8b5-c063ca053e6e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:57.541892 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-cb15416f-92dc-4d64-a8b5-c063ca053e6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7df6740c4254f4692fe72d60c9dcaf5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:57.542545 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-cb15416f-92dc-4d64-a8b5-c063ca053e6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7df6740c4254f4692fe72d60c9dcaf5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:57.543190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-cb15416f-92dc-4d64-a8b5-c063ca053e6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7df6740c4254f4692fe72d60c9dcaf5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:57.550518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-cb15416f-92dc-4d64-a8b5-c063ca053e6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7df6740c4254f4692fe72d60c9dcaf5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:57.561987 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-cb15416f-92dc-4d64-a8b5-c063ca053e6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7df6740c4254f4692fe72d60c9dcaf5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:57.578608 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1379/2784] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:26:57 2022] DELETE /identity/v3/users/b7df6740c4254f4692fe72d60c9dcaf5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:26:57.583348 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6baf523-ed8b-48a7-ba64-e5cd2a022048 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:57.583843 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6baf523-ed8b-48a7-ba64-e5cd2a022048 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:57.583965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6baf523-ed8b-48a7-ba64-e5cd2a022048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:57.605731 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-e6baf523-ed8b-48a7-ba64-e5cd2a022048 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:26:57.609142 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e6baf523-ed8b-48a7-ba64-e5cd2a022048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:26:57.686918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1406/2785] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:26:57.697568 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15416f-92dc-4d64-a8b5-c063ca053e6e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:57.700099 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-cb15416f-92dc-4d64-a8b5-c063ca053e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:26:57.762486 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15416f-92dc-4d64-a8b5-c063ca053e6e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:57.767378 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494e2b3-421d-4325-b280-918e370dd56e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:57.768024 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494e2b3-421d-4325-b280-918e370dd56e service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:57.768302 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494e2b3-421d-4325-b280-918e370dd56e service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:57.768569 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494e2b3-421d-4325-b280-918e370dd56e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:57.778968 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494e2b3-421d-4325-b280-918e370dd56e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:57.780451 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494e2b3-421d-4325-b280-918e370dd56e service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:57.790854 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1380/2786] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:26:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:26:58.070498 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6baf523-ed8b-48a7-ba64-e5cd2a022048 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:58.072188 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-e6baf523-ed8b-48a7-ba64-e5cd2a022048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:26:58.087214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6baf523-ed8b-48a7-ba64-e5cd2a022048 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:58.092685 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3951405-8b88-4159-b453-9f69edb0fbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:58.093448 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3951405-8b88-4159-b453-9f69edb0fbf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:58.093783 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3951405-8b88-4159-b453-9f69edb0fbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:58.094123 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3951405-8b88-4159-b453-9f69edb0fbf1 admin admin] PATH_INFO: `/v3/projects/f744a0f1eb624db1aeb75dab7a81323b` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:58.095898 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3951405-8b88-4159-b453-9f69edb0fbf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f744a0f1eb624db1aeb75dab7a81323b)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:58.101077 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3951405-8b88-4159-b453-9f69edb0fbf1 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:58.161046 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c3951405-8b88-4159-b453-9f69edb0fbf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f744a0f1eb624db1aeb75dab7a81323b', 'request_id': 'req-c3951405-8b88-4159-b453-9f69edb0fbf1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:58.167112 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c3951405-8b88-4159-b453-9f69edb0fbf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f744a0f1eb624db1aeb75dab7a81323b', 'request_id': 'req-c3951405-8b88-4159-b453-9f69edb0fbf1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:58.176816 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c3951405-8b88-4159-b453-9f69edb0fbf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f744a0f1eb624db1aeb75dab7a81323b', 'request_id': 'req-c3951405-8b88-4159-b453-9f69edb0fbf1'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:58.178207 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:26:58.191441 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c3951405-8b88-4159-b453-9f69edb0fbf1 admin admin] The token cache is being invalidate because project f744a0f1eb624db1aeb75dab7a81323b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:26:58.191747 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c3951405-8b88-4159-b453-9f69edb0fbf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:58.192451 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-c3951405-8b88-4159-b453-9f69edb0fbf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:58.193986 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1407/2787] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:26:58 2022] DELETE /identity/v3/projects/f744a0f1eb624db1aeb75dab7a81323b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:26:58.198190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494e2b3-421d-4325-b280-918e370dd56e service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:58.208130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494e2b3-421d-4325-b280-918e370dd56e service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:58.238172 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:26:58.268664 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643f866-ab44-4a25-8a1d-59f941b350e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:58.269271 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643f866-ab44-4a25-8a1d-59f941b350e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:58.269568 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643f866-ab44-4a25-8a1d-59f941b350e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:58.269840 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643f866-ab44-4a25-8a1d-59f941b350e5 admin admin] PATH_INFO: `/v3/users/28e1f66e0d094cd1a23d73f38c0852ff` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:58.271083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643f866-ab44-4a25-8a1d-59f941b350e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28e1f66e0d094cd1a23d73f38c0852ff)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:58.273049 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643f866-ab44-4a25-8a1d-59f941b350e5 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:58.323305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4643f866-ab44-4a25-8a1d-59f941b350e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28e1f66e0d094cd1a23d73f38c0852ff'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:58.324105 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4643f866-ab44-4a25-8a1d-59f941b350e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28e1f66e0d094cd1a23d73f38c0852ff'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:58.324825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4643f866-ab44-4a25-8a1d-59f941b350e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28e1f66e0d094cd1a23d73f38c0852ff'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:58.332916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4643f866-ab44-4a25-8a1d-59f941b350e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28e1f66e0d094cd1a23d73f38c0852ff'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:58.344867 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4643f866-ab44-4a25-8a1d-59f941b350e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28e1f66e0d094cd1a23d73f38c0852ff'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:58.359438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1381/2788] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:26:58 2022] DELETE /identity/v3/users/28e1f66e0d094cd1a23d73f38c0852ff => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:26:58.648484 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3951405-8b88-4159-b453-9f69edb0fbf1 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:58.650090 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c3951405-8b88-4159-b453-9f69edb0fbf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:26:58.664886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3951405-8b88-4159-b453-9f69edb0fbf1 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:58.724413 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4a720-e2d9-499c-aae0-e080d104d95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:58.725049 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4a720-e2d9-499c-aae0-e080d104d95b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:58.725331 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4a720-e2d9-499c-aae0-e080d104d95b admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:58.725601 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4a720-e2d9-499c-aae0-e080d104d95b admin admin] PATH_INFO: `/v3/projects/3545782601024658a00a6904f7e22d1a` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:58.726752 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4a720-e2d9-499c-aae0-e080d104d95b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3545782601024658a00a6904f7e22d1a)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:58.728726 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4a720-e2d9-499c-aae0-e080d104d95b admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:58.780709 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-0ae4a720-e2d9-499c-aae0-e080d104d95b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3545782601024658a00a6904f7e22d1a', 'request_id': 'req-0ae4a720-e2d9-499c-aae0-e080d104d95b'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:58.787566 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-0ae4a720-e2d9-499c-aae0-e080d104d95b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3545782601024658a00a6904f7e22d1a', 'request_id': 'req-0ae4a720-e2d9-499c-aae0-e080d104d95b'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:58.797537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-0ae4a720-e2d9-499c-aae0-e080d104d95b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3545782601024658a00a6904f7e22d1a', 'request_id': 'req-0ae4a720-e2d9-499c-aae0-e080d104d95b'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:58.809467 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-0ae4a720-e2d9-499c-aae0-e080d104d95b admin admin] The token cache is being invalidate because project 3545782601024658a00a6904f7e22d1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:26:58.809773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-0ae4a720-e2d9-499c-aae0-e080d104d95b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:58.810505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-0ae4a720-e2d9-499c-aae0-e080d104d95b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:26:58.812147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1408/2789] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:26:58 2022] DELETE /identity/v3/projects/3545782601024658a00a6904f7e22d1a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:26:58.817365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a167f1-c6ca-4d7a-82d1-36567cc8e451 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:58.817878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a167f1-c6ca-4d7a-82d1-36567cc8e451 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:58.817878 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a167f1-c6ca-4d7a-82d1-36567cc8e451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:58.838743 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-11a167f1-c6ca-4d7a-82d1-36567cc8e451 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:26:58.839928 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-11a167f1-c6ca-4d7a-82d1-36567cc8e451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:26:58.901459 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1382/2790] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:26:58.910664 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba4af6-8b57-405c-809b-7adb43ffa102 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:58.910949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba4af6-8b57-405c-809b-7adb43ffa102 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:58.911211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba4af6-8b57-405c-809b-7adb43ffa102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:58.932987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-91ba4af6-8b57-405c-809b-7adb43ffa102 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:26:58.934802 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-91ba4af6-8b57-405c-809b-7adb43ffa102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:26:58.948964 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1409/2791] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:26:58.954148 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a167f1-c6ca-4d7a-82d1-36567cc8e451 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:58.964564 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a167f1-c6ca-4d7a-82d1-36567cc8e451 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:58.969817 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af8b31-f957-440c-ae37-de6e27d93d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:58.970654 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af8b31-f957-440c-ae37-de6e27d93d8c admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:58.970936 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af8b31-f957-440c-ae37-de6e27d93d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:58.971215 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af8b31-f957-440c-ae37-de6e27d93d8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:58.971900 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af8b31-f957-440c-ae37-de6e27d93d8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:26:58.972254 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af8b31-f957-440c-ae37-de6e27d93d8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:58.974215 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af8b31-f957-440c-ae37-de6e27d93d8c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:58.986540 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1383/2792] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:26:58.991288 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba4af6-8b57-405c-809b-7adb43ffa102 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:59.002355 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba4af6-8b57-405c-809b-7adb43ffa102 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:59.012099 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfc10c-74bc-4dfb-b168-be6c00872529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:59.012990 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfc10c-74bc-4dfb-b168-be6c00872529 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:59.013320 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfc10c-74bc-4dfb-b168-be6c00872529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:59.013706 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfc10c-74bc-4dfb-b168-be6c00872529 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:59.014592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bfc10c-74bc-4dfb-b168-be6c00872529 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:59.017380 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bfc10c-74bc-4dfb-b168-be6c00872529 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:59.047760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1410/2793] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:26:58 2022] POST /identity/v3/projects => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:26:59.054292 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af8b31-f957-440c-ae37-de6e27d93d8c admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:59.071855 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af8b31-f957-440c-ae37-de6e27d93d8c admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:59.108730 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:26:59.134883 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabae6a3-b3ac-44d4-b8cf-e8e04d2e20b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:59.135530 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabae6a3-b3ac-44d4-b8cf-e8e04d2e20b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:59.135813 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabae6a3-b3ac-44d4-b8cf-e8e04d2e20b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:59.136087 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabae6a3-b3ac-44d4-b8cf-e8e04d2e20b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:59.136717 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabae6a3-b3ac-44d4-b8cf-e8e04d2e20b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:59.138661 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabae6a3-b3ac-44d4-b8cf-e8e04d2e20b6 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:59.160070 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1384/2794] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:26:59 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:26:59.164329 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfc10c-74bc-4dfb-b168-be6c00872529 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:59.175338 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfc10c-74bc-4dfb-b168-be6c00872529 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:59.181179 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c41dc6-3dcf-4a29-aa49-72bcfbc5fddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:59.181801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c41dc6-3dcf-4a29-aa49-72bcfbc5fddc admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:59.182078 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c41dc6-3dcf-4a29-aa49-72bcfbc5fddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:59.182346 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c41dc6-3dcf-4a29-aa49-72bcfbc5fddc admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:59.183018 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c41dc6-3dcf-4a29-aa49-72bcfbc5fddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:59.184663 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c41dc6-3dcf-4a29-aa49-72bcfbc5fddc admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:59.195510 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1411/2795] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:26:59.199688 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabae6a3-b3ac-44d4-b8cf-e8e04d2e20b6 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:59.210801 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabae6a3-b3ac-44d4-b8cf-e8e04d2e20b6 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:59.216553 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b67e0-652a-4ac9-b85d-5ed2e9b10cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:59.217147 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b67e0-652a-4ac9-b85d-5ed2e9b10cfb admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:59.217419 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b67e0-652a-4ac9-b85d-5ed2e9b10cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:59.217693 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b67e0-652a-4ac9-b85d-5ed2e9b10cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:59.218337 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b67e0-652a-4ac9-b85d-5ed2e9b10cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:59.220037 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b67e0-652a-4ac9-b85d-5ed2e9b10cfb admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:59.230629 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1385/2796] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:26:59.235042 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c41dc6-3dcf-4a29-aa49-72bcfbc5fddc admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:59.246667 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c41dc6-3dcf-4a29-aa49-72bcfbc5fddc admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:59.252052 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c23351-df2b-4ef2-aa0e-b4123e27ce6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:59.252742 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c23351-df2b-4ef2-aa0e-b4123e27ce6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:59.253063 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c23351-df2b-4ef2-aa0e-b4123e27ce6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:59.253365 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c23351-df2b-4ef2-aa0e-b4123e27ce6e admin admin] PATH_INFO: `/v3/projects/14b9acc00b904762b24d80e51ae737e6/users/5746639440da4bd88c4805cc98ada313/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:59.269276 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c23351-df2b-4ef2-aa0e-b4123e27ce6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=14b9acc00b904762b24d80e51ae737e6, user_id=5746639440da4bd88c4805cc98ada313, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:59.271993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c23351-df2b-4ef2-aa0e-b4123e27ce6e admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:59.287440 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1412/2797] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:26:59 2022] PUT /identity/v3/projects/14b9acc00b904762b24d80e51ae737e6/users/5746639440da4bd88c4805cc98ada313/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:26:59.299740 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f5941-0d0a-4321-b721-99fd59c51df8 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:59.300106 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f5941-0d0a-4321-b721-99fd59c51df8 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:59.300384 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f5941-0d0a-4321-b721-99fd59c51df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:59.331584 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-3c7f5941-0d0a-4321-b721-99fd59c51df8 None None] MFA Rules not processed for user `5746639440da4bd88c4805cc98ada313`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:26:59.332785 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-3c7f5941-0d0a-4321-b721-99fd59c51df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:26:59.406609 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1386/2798] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:26:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:26:59.413413 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c23351-df2b-4ef2-aa0e-b4123e27ce6e admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:59.439554 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:26:59.482215 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c23351-df2b-4ef2-aa0e-b4123e27ce6e admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:59.487237 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b59ef-095f-47b6-b621-fc1947efa041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:59.487896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b59ef-095f-47b6-b621-fc1947efa041 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:59.488205 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b59ef-095f-47b6-b621-fc1947efa041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:59.488475 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b59ef-095f-47b6-b621-fc1947efa041 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:59.489110 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b59ef-095f-47b6-b621-fc1947efa041 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:59.491013 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b59ef-095f-47b6-b621-fc1947efa041 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:59.520417 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1413/2799] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:26:59 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:26:59.524704 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f5941-0d0a-4321-b721-99fd59c51df8 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:59.535976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f5941-0d0a-4321-b721-99fd59c51df8 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:59.594617 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ae929-9fec-40a1-966f-1111e22ba2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:59.595255 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ae929-9fec-40a1-966f-1111e22ba2db admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:59.595568 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ae929-9fec-40a1-966f-1111e22ba2db admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:59.595849 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ae929-9fec-40a1-966f-1111e22ba2db admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:59.596489 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ae929-9fec-40a1-966f-1111e22ba2db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:59.598764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ae929-9fec-40a1-966f-1111e22ba2db admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:59.622522 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1387/2800] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:26:59 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:26:59.626961 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b59ef-095f-47b6-b621-fc1947efa041 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:59.637332 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b59ef-095f-47b6-b621-fc1947efa041 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:59.642172 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7688e13-5ed1-4822-80ae-93db7b1119ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:59.642769 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7688e13-5ed1-4822-80ae-93db7b1119ab admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:59.643051 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7688e13-5ed1-4822-80ae-93db7b1119ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:59.643325 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7688e13-5ed1-4822-80ae-93db7b1119ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:59.644071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7688e13-5ed1-4822-80ae-93db7b1119ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:59.645671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7688e13-5ed1-4822-80ae-93db7b1119ab admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:59.656840 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1414/2801] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:26:59.661158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ae929-9fec-40a1-966f-1111e22ba2db admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:59.671701 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ae929-9fec-40a1-966f-1111e22ba2db admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:59.677880 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ac392-0379-4667-81f6-d028bbef2ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:59.678532 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ac392-0379-4667-81f6-d028bbef2ca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:59.678808 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ac392-0379-4667-81f6-d028bbef2ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:59.679078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ac392-0379-4667-81f6-d028bbef2ca0 admin admin] PATH_INFO: `/v3/projects/2ac67a1cba56402d8f2cd279f156afea/users/092b1f7b5f2c4ac1adf56b9c014770ee/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:59.698328 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ac392-0379-4667-81f6-d028bbef2ca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac67a1cba56402d8f2cd279f156afea, user_id=092b1f7b5f2c4ac1adf56b9c014770ee, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:59.701043 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ac392-0379-4667-81f6-d028bbef2ca0 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:59.716570 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1388/2802] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:26:59 2022] PUT /identity/v3/projects/2ac67a1cba56402d8f2cd279f156afea/users/092b1f7b5f2c4ac1adf56b9c014770ee/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:26:59.720669 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7688e13-5ed1-4822-80ae-93db7b1119ab admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:59.731262 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7688e13-5ed1-4822-80ae-93db7b1119ab admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:59.787367 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36572d8-ea20-4b5c-b0ac-17e191d5a91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:59.788019 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36572d8-ea20-4b5c-b0ac-17e191d5a91d admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:59.788302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36572d8-ea20-4b5c-b0ac-17e191d5a91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:59.788575 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36572d8-ea20-4b5c-b0ac-17e191d5a91d admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:59.789214 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36572d8-ea20-4b5c-b0ac-17e191d5a91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:59.790855 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36572d8-ea20-4b5c-b0ac-17e191d5a91d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:59.800758 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1415/2803] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:26:59.805092 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ac392-0379-4667-81f6-d028bbef2ca0 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:59.815764 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ac392-0379-4667-81f6-d028bbef2ca0 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:26:59.820719 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11afcc-c4f9-4581-a34b-27c351b4f312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:26:59.821387 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11afcc-c4f9-4581-a34b-27c351b4f312 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:59.821664 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11afcc-c4f9-4581-a34b-27c351b4f312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:59.821932 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11afcc-c4f9-4581-a34b-27c351b4f312 admin admin] PATH_INFO: `/v3/projects/2ac67a1cba56402d8f2cd279f156afea/users/092b1f7b5f2c4ac1adf56b9c014770ee/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:59.823937 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11afcc-c4f9-4581-a34b-27c351b4f312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac67a1cba56402d8f2cd279f156afea, user_id=092b1f7b5f2c4ac1adf56b9c014770ee, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:26:59.826848 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11afcc-c4f9-4581-a34b-27c351b4f312 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:26:59.833148 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:26:59.838669 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1389/2804] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:26:59 2022] PUT /identity/v3/projects/2ac67a1cba56402d8f2cd279f156afea/users/092b1f7b5f2c4ac1adf56b9c014770ee/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:26:59.849388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5553d0a-f359-456f-ad71-bb730de63d53 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:26:59.849675 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5553d0a-f359-456f-ad71-bb730de63d53 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:26:59.849952 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5553d0a-f359-456f-ad71-bb730de63d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:26:59.877968 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-b5553d0a-f359-456f-ad71-bb730de63d53 None None] MFA Rules not processed for user `092b1f7b5f2c4ac1adf56b9c014770ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:26:59.879187 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b5553d0a-f359-456f-ad71-bb730de63d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:26:59.958827 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1416/2805] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:26:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:26:59.971065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11afcc-c4f9-4581-a34b-27c351b4f312 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:26:59.974335 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-0d11afcc-c4f9-4581-a34b-27c351b4f312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:00.038611 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11afcc-c4f9-4581-a34b-27c351b4f312 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:00.042509 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb165e-fc69-435b-8d53-621fa041517e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:00.043110 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb165e-fc69-435b-8d53-621fa041517e service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:00.043401 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb165e-fc69-435b-8d53-621fa041517e service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:00.043683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb165e-fc69-435b-8d53-621fa041517e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:00.052956 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb165e-fc69-435b-8d53-621fa041517e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:00.054391 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb165e-fc69-435b-8d53-621fa041517e service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:00.064132 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1390/2806] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:26:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:27:00.110989 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5553d0a-f359-456f-ad71-bb730de63d53 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:00.112671 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b5553d0a-f359-456f-ad71-bb730de63d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:00.125344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5553d0a-f359-456f-ad71-bb730de63d53 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:00.129696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b800a8-630a-480c-89e5-5022c4913221 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:00.130295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b800a8-630a-480c-89e5-5022c4913221 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:00.130578 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b800a8-630a-480c-89e5-5022c4913221 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:00.130853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b800a8-630a-480c-89e5-5022c4913221 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:00.144650 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:00.196186 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b800a8-630a-480c-89e5-5022c4913221 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:00.197621 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b800a8-630a-480c-89e5-5022c4913221 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:00.208401 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1417/2807] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:27:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:27:03.621024 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb165e-fc69-435b-8d53-621fa041517e service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:03.682476 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb165e-fc69-435b-8d53-621fa041517e service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:03.687100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04f575-f790-4070-a4ee-6554dee80e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:03.687745 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04f575-f790-4070-a4ee-6554dee80e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:03.688018 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04f575-f790-4070-a4ee-6554dee80e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:03.688285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04f575-f790-4070-a4ee-6554dee80e6a admin admin] PATH_INFO: `/v3/users/5746639440da4bd88c4805cc98ada313` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:03.689633 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04f575-f790-4070-a4ee-6554dee80e6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5746639440da4bd88c4805cc98ada313)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:03.691601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04f575-f790-4070-a4ee-6554dee80e6a admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:03.744370 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1d04f575-f790-4070-a4ee-6554dee80e6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5746639440da4bd88c4805cc98ada313'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:03.745072 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1d04f575-f790-4070-a4ee-6554dee80e6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5746639440da4bd88c4805cc98ada313'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:03.745722 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1d04f575-f790-4070-a4ee-6554dee80e6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5746639440da4bd88c4805cc98ada313'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:03.753172 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1d04f575-f790-4070-a4ee-6554dee80e6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5746639440da4bd88c4805cc98ada313'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:03.765855 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-1d04f575-f790-4070-a4ee-6554dee80e6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5746639440da4bd88c4805cc98ada313'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:03.779456 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1391/2808] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:27:03 2022] DELETE /identity/v3/users/5746639440da4bd88c4805cc98ada313 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:03.784519 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caae743-13ce-4632-a882-e52bb60080af None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:03.785294 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caae743-13ce-4632-a882-e52bb60080af None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:03.785463 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caae743-13ce-4632-a882-e52bb60080af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:03.805333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-3caae743-13ce-4632-a882-e52bb60080af None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:03.806455 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3caae743-13ce-4632-a882-e52bb60080af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:03.873280 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1418/2809] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:03.883974 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04f575-f790-4070-a4ee-6554dee80e6a admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:03.885588 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-1d04f575-f790-4070-a4ee-6554dee80e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:03.914886 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:03.955032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04f575-f790-4070-a4ee-6554dee80e6a admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:03.959631 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630c1e1-23f6-4135-962b-273a08df7638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:03.960236 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630c1e1-23f6-4135-962b-273a08df7638 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:03.960518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630c1e1-23f6-4135-962b-273a08df7638 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:03.960796 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630c1e1-23f6-4135-962b-273a08df7638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:03.970746 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630c1e1-23f6-4135-962b-273a08df7638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:03.972270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630c1e1-23f6-4135-962b-273a08df7638 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:03.982305 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1392/2810] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:27:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:27:04.270907 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caae743-13ce-4632-a882-e52bb60080af None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:04.272398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-3caae743-13ce-4632-a882-e52bb60080af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:04.284859 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caae743-13ce-4632-a882-e52bb60080af None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:04.290317 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7b09a-1c10-4e1e-879e-1ed16ad9a4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:04.290988 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7b09a-1c10-4e1e-879e-1ed16ad9a4d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:04.291277 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7b09a-1c10-4e1e-879e-1ed16ad9a4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:04.291577 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7b09a-1c10-4e1e-879e-1ed16ad9a4d5 admin admin] PATH_INFO: `/v3/projects/14b9acc00b904762b24d80e51ae737e6` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:04.292819 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7b09a-1c10-4e1e-879e-1ed16ad9a4d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14b9acc00b904762b24d80e51ae737e6)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:04.294762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7b09a-1c10-4e1e-879e-1ed16ad9a4d5 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:04.361195 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-66f7b09a-1c10-4e1e-879e-1ed16ad9a4d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14b9acc00b904762b24d80e51ae737e6', 'request_id': 'req-66f7b09a-1c10-4e1e-879e-1ed16ad9a4d5'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:04.368270 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-66f7b09a-1c10-4e1e-879e-1ed16ad9a4d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14b9acc00b904762b24d80e51ae737e6', 'request_id': 'req-66f7b09a-1c10-4e1e-879e-1ed16ad9a4d5'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:04.380858 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-66f7b09a-1c10-4e1e-879e-1ed16ad9a4d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14b9acc00b904762b24d80e51ae737e6', 'request_id': 'req-66f7b09a-1c10-4e1e-879e-1ed16ad9a4d5'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:04.396206 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-66f7b09a-1c10-4e1e-879e-1ed16ad9a4d5 admin admin] The token cache is being invalidate because project 14b9acc00b904762b24d80e51ae737e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:27:04.396700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-66f7b09a-1c10-4e1e-879e-1ed16ad9a4d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:04.397433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-66f7b09a-1c10-4e1e-879e-1ed16ad9a4d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:04.400050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1419/2811] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:27:04 2022] DELETE /identity/v3/projects/14b9acc00b904762b24d80e51ae737e6 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:04.406285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630c1e1-23f6-4135-962b-273a08df7638 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:04.421705 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630c1e1-23f6-4135-962b-273a08df7638 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:04.480323 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa5af8-adb2-4dd2-812a-1329060fa63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:04.480923 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa5af8-adb2-4dd2-812a-1329060fa63f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:04.481196 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa5af8-adb2-4dd2-812a-1329060fa63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:04.481463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa5af8-adb2-4dd2-812a-1329060fa63f admin admin] PATH_INFO: `/v3/users/092b1f7b5f2c4ac1adf56b9c014770ee` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:04.482552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa5af8-adb2-4dd2-812a-1329060fa63f admin admin] RBAC: Authorizing `identity:delete_user(user_id=092b1f7b5f2c4ac1adf56b9c014770ee)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:04.484521 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa5af8-adb2-4dd2-812a-1329060fa63f admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:04.535945 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e6aa5af8-adb2-4dd2-812a-1329060fa63f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '092b1f7b5f2c4ac1adf56b9c014770ee'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:04.536648 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e6aa5af8-adb2-4dd2-812a-1329060fa63f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '092b1f7b5f2c4ac1adf56b9c014770ee'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:04.537304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e6aa5af8-adb2-4dd2-812a-1329060fa63f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '092b1f7b5f2c4ac1adf56b9c014770ee'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:04.547065 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e6aa5af8-adb2-4dd2-812a-1329060fa63f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '092b1f7b5f2c4ac1adf56b9c014770ee'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:04.565786 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-e6aa5af8-adb2-4dd2-812a-1329060fa63f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '092b1f7b5f2c4ac1adf56b9c014770ee'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:04.581365 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1393/2812] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:27:04 2022] DELETE /identity/v3/users/092b1f7b5f2c4ac1adf56b9c014770ee => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:04.865120 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7b09a-1c10-4e1e-879e-1ed16ad9a4d5 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:04.867251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-66f7b09a-1c10-4e1e-879e-1ed16ad9a4d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:04.881027 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7b09a-1c10-4e1e-879e-1ed16ad9a4d5 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:04.917747 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:04.942937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de611a71-a62f-4480-bfa1-60db4da6be1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:04.943611 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de611a71-a62f-4480-bfa1-60db4da6be1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:04.943894 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de611a71-a62f-4480-bfa1-60db4da6be1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:04.944165 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de611a71-a62f-4480-bfa1-60db4da6be1e admin admin] PATH_INFO: `/v3/projects/2ac67a1cba56402d8f2cd279f156afea` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:04.945382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de611a71-a62f-4480-bfa1-60db4da6be1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ac67a1cba56402d8f2cd279f156afea)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:04.947336 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de611a71-a62f-4480-bfa1-60db4da6be1e admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:04.997592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-de611a71-a62f-4480-bfa1-60db4da6be1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ac67a1cba56402d8f2cd279f156afea', 'request_id': 'req-de611a71-a62f-4480-bfa1-60db4da6be1e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:05.003202 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-de611a71-a62f-4480-bfa1-60db4da6be1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ac67a1cba56402d8f2cd279f156afea', 'request_id': 'req-de611a71-a62f-4480-bfa1-60db4da6be1e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:05.011686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-de611a71-a62f-4480-bfa1-60db4da6be1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ac67a1cba56402d8f2cd279f156afea', 'request_id': 'req-de611a71-a62f-4480-bfa1-60db4da6be1e'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:05.022934 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-de611a71-a62f-4480-bfa1-60db4da6be1e admin admin] The token cache is being invalidate because project 2ac67a1cba56402d8f2cd279f156afea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:27:05.023233 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-de611a71-a62f-4480-bfa1-60db4da6be1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:05.023883 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-de611a71-a62f-4480-bfa1-60db4da6be1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:05.025375 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1420/2813] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:27:04 2022] DELETE /identity/v3/projects/2ac67a1cba56402d8f2cd279f156afea => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:05.030460 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b731c-394f-400e-aaa3-69aface64569 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:05.030978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b731c-394f-400e-aaa3-69aface64569 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:05.030978 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b731c-394f-400e-aaa3-69aface64569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:05.051379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-839b731c-394f-400e-aaa3-69aface64569 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:05.052766 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-839b731c-394f-400e-aaa3-69aface64569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:05.092347 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:05.115737 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1394/2814] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:05.124781 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d34f6-a011-4365-a123-e97449123737 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:05.125067 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d34f6-a011-4365-a123-e97449123737 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:05.125340 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d34f6-a011-4365-a123-e97449123737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:05.146075 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-c54d34f6-a011-4365-a123-e97449123737 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:05.147289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-c54d34f6-a011-4365-a123-e97449123737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:05.155944 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1421/2815] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:05.160363 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b731c-394f-400e-aaa3-69aface64569 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:05.170494 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b731c-394f-400e-aaa3-69aface64569 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:05.176015 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa11cbd-0d9e-420d-ab8e-43467654256b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:05.176617 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa11cbd-0d9e-420d-ab8e-43467654256b admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:05.176915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa11cbd-0d9e-420d-ab8e-43467654256b admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:05.177185 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa11cbd-0d9e-420d-ab8e-43467654256b admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:05.177854 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa11cbd-0d9e-420d-ab8e-43467654256b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:27:05.178156 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa11cbd-0d9e-420d-ab8e-43467654256b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:05.179805 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa11cbd-0d9e-420d-ab8e-43467654256b admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:05.191751 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1395/2816] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:27:05.196166 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d34f6-a011-4365-a123-e97449123737 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:05.206094 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d34f6-a011-4365-a123-e97449123737 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:05.211501 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af8079-445a-477c-a395-df7417759682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:05.212132 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af8079-445a-477c-a395-df7417759682 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:05.212413 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af8079-445a-477c-a395-df7417759682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:05.212683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af8079-445a-477c-a395-df7417759682 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:05.213305 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af8079-445a-477c-a395-df7417759682 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:05.215221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af8079-445a-477c-a395-df7417759682 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:05.244409 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1422/2817] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:27:05 2022] POST /identity/v3/projects => generated 409 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:27:05.248669 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa11cbd-0d9e-420d-ab8e-43467654256b admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:05.259007 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa11cbd-0d9e-420d-ab8e-43467654256b admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:05.319655 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc8944-24e5-49fb-8589-00d3b8551114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:05.320271 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc8944-24e5-49fb-8589-00d3b8551114 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:05.320549 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc8944-24e5-49fb-8589-00d3b8551114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:05.320820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc8944-24e5-49fb-8589-00d3b8551114 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:05.321441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc8944-24e5-49fb-8589-00d3b8551114 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:05.323443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc8944-24e5-49fb-8589-00d3b8551114 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:05.346046 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1396/2818] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:27:05 2022] POST /identity/v3/users => generated 305 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:27:05.350712 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af8079-445a-477c-a395-df7417759682 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:05.361637 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af8079-445a-477c-a395-df7417759682 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:05.367572 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99ad53-8552-460c-bcf4-a8752677b010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:05.368173 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99ad53-8552-460c-bcf4-a8752677b010 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:05.368432 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99ad53-8552-460c-bcf4-a8752677b010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:05.368705 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99ad53-8552-460c-bcf4-a8752677b010 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:05.369332 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99ad53-8552-460c-bcf4-a8752677b010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:05.370937 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99ad53-8552-460c-bcf4-a8752677b010 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:05.381505 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1423/2819] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:27:05.387664 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc8944-24e5-49fb-8589-00d3b8551114 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:05.402288 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc8944-24e5-49fb-8589-00d3b8551114 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:05.409239 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81391fb8-6a11-4275-88b9-ccbe10c973cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:05.409849 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81391fb8-6a11-4275-88b9-ccbe10c973cb admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:05.410130 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81391fb8-6a11-4275-88b9-ccbe10c973cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:05.410403 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81391fb8-6a11-4275-88b9-ccbe10c973cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:05.411052 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81391fb8-6a11-4275-88b9-ccbe10c973cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:05.412762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81391fb8-6a11-4275-88b9-ccbe10c973cb admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:05.424598 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1397/2820] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:27:05.428869 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99ad53-8552-460c-bcf4-a8752677b010 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:05.439826 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99ad53-8552-460c-bcf4-a8752677b010 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:05.445333 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca532290-191a-410b-9c66-8a0df8a92c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:05.445979 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca532290-191a-410b-9c66-8a0df8a92c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:05.446267 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca532290-191a-410b-9c66-8a0df8a92c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:05.446537 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca532290-191a-410b-9c66-8a0df8a92c0f admin admin] PATH_INFO: `/v3/projects/08818811798f480e9b10300a0390ce80/users/c6049d91d123462eac8bb251461c3354/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:05.467670 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca532290-191a-410b-9c66-8a0df8a92c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=08818811798f480e9b10300a0390ce80, user_id=c6049d91d123462eac8bb251461c3354, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:05.471156 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca532290-191a-410b-9c66-8a0df8a92c0f admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:05.486165 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1424/2821] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:27:05 2022] PUT /identity/v3/projects/08818811798f480e9b10300a0390ce80/users/c6049d91d123462eac8bb251461c3354/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:27:05.496548 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c93390-19ed-438b-b73a-8d45e8cec184 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:05.496830 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c93390-19ed-438b-b73a-8d45e8cec184 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:05.497102 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c93390-19ed-438b-b73a-8d45e8cec184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:05.524443 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-e8c93390-19ed-438b-b73a-8d45e8cec184 None None] MFA Rules not processed for user `c6049d91d123462eac8bb251461c3354`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:05.525641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-e8c93390-19ed-438b-b73a-8d45e8cec184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:05.588070 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1398/2822] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:05 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:05.592821 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca532290-191a-410b-9c66-8a0df8a92c0f admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:05.665765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca532290-191a-410b-9c66-8a0df8a92c0f admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:05.671598 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f36762-e016-42a7-922f-1a3392e96f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:05.672201 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f36762-e016-42a7-922f-1a3392e96f08 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:05.672489 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f36762-e016-42a7-922f-1a3392e96f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:05.672765 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f36762-e016-42a7-922f-1a3392e96f08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:05.673604 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f36762-e016-42a7-922f-1a3392e96f08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:05.675652 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f36762-e016-42a7-922f-1a3392e96f08 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:05.685443 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:05.705760 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1425/2823] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:27:05 2022] POST /identity/v3/projects => generated 409 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:27:05.710103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c93390-19ed-438b-b73a-8d45e8cec184 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:05.721556 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c93390-19ed-438b-b73a-8d45e8cec184 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:05.757921 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:05.783401 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8036f-b23d-4c8a-aa16-5142e93913c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:05.784035 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8036f-b23d-4c8a-aa16-5142e93913c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:05.784316 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8036f-b23d-4c8a-aa16-5142e93913c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:05.784583 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8036f-b23d-4c8a-aa16-5142e93913c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:05.785219 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8036f-b23d-4c8a-aa16-5142e93913c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:05.787151 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8036f-b23d-4c8a-aa16-5142e93913c0 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:05.810361 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1399/2824] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:27:05 2022] POST /identity/v3/users => generated 311 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:27:05.814647 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f36762-e016-42a7-922f-1a3392e96f08 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:05.825253 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f36762-e016-42a7-922f-1a3392e96f08 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:05.830198 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f69d0-6cb9-4da5-a070-73f2458f33b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:05.830779 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f69d0-6cb9-4da5-a070-73f2458f33b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:05.831063 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f69d0-6cb9-4da5-a070-73f2458f33b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:05.831328 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f69d0-6cb9-4da5-a070-73f2458f33b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:05.832007 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f69d0-6cb9-4da5-a070-73f2458f33b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:05.833590 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f69d0-6cb9-4da5-a070-73f2458f33b2 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:05.844432 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1426/2825] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:27:05.849147 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8036f-b23d-4c8a-aa16-5142e93913c0 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:05.859641 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8036f-b23d-4c8a-aa16-5142e93913c0 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:05.865455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ae4fb-512c-4f40-ab6d-1fc9dca358f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:05.866094 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ae4fb-512c-4f40-ab6d-1fc9dca358f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:05.866369 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ae4fb-512c-4f40-ab6d-1fc9dca358f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:05.866645 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ae4fb-512c-4f40-ab6d-1fc9dca358f9 admin admin] PATH_INFO: `/v3/projects/4c3c25c09c244a289d1a524c20966997/users/96744928634a452bb9c60999ee5c3792/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:05.882010 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ae4fb-512c-4f40-ab6d-1fc9dca358f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c3c25c09c244a289d1a524c20966997, user_id=96744928634a452bb9c60999ee5c3792, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:05.884721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ae4fb-512c-4f40-ab6d-1fc9dca358f9 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:05.901412 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1400/2826] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:27:05 2022] PUT /identity/v3/projects/4c3c25c09c244a289d1a524c20966997/users/96744928634a452bb9c60999ee5c3792/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:05.907545 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f69d0-6cb9-4da5-a070-73f2458f33b2 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:05.922821 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f69d0-6cb9-4da5-a070-73f2458f33b2 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:05.982866 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e1509-a9b9-46e0-9074-49a96f5f6b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:05.983396 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e1509-a9b9-46e0-9074-49a96f5f6b55 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:05.983705 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e1509-a9b9-46e0-9074-49a96f5f6b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:05.983977 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e1509-a9b9-46e0-9074-49a96f5f6b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:05.984617 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e1509-a9b9-46e0-9074-49a96f5f6b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:05.986288 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e1509-a9b9-46e0-9074-49a96f5f6b55 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:05.996113 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1427/2827] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:27:06.000441 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ae4fb-512c-4f40-ab6d-1fc9dca358f9 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:06.012071 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ae4fb-512c-4f40-ab6d-1fc9dca358f9 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:06.017422 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768c109-76ba-4ccd-87bd-337fd8349b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:06.018069 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768c109-76ba-4ccd-87bd-337fd8349b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:06.018340 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768c109-76ba-4ccd-87bd-337fd8349b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:06.018607 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768c109-76ba-4ccd-87bd-337fd8349b8e admin admin] PATH_INFO: `/v3/projects/4c3c25c09c244a289d1a524c20966997/users/96744928634a452bb9c60999ee5c3792/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:06.020518 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768c109-76ba-4ccd-87bd-337fd8349b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c3c25c09c244a289d1a524c20966997, user_id=96744928634a452bb9c60999ee5c3792, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:06.023154 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768c109-76ba-4ccd-87bd-337fd8349b8e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:06.035583 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1401/2828] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:27:05 2022] PUT /identity/v3/projects/4c3c25c09c244a289d1a524c20966997/users/96744928634a452bb9c60999ee5c3792/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:06.047259 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fd360-c800-4f6c-a305-ef8995b5b021 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:06.047743 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fd360-c800-4f6c-a305-ef8995b5b021 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:06.048081 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fd360-c800-4f6c-a305-ef8995b5b021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:06.082738 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-979fd360-c800-4f6c-a305-ef8995b5b021 None None] MFA Rules not processed for user `96744928634a452bb9c60999ee5c3792`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:06.083993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-979fd360-c800-4f6c-a305-ef8995b5b021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:06.166344 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1428/2829] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:06 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:06.177509 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768c109-76ba-4ccd-87bd-337fd8349b8e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:06.179508 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-0768c109-76ba-4ccd-87bd-337fd8349b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:06.243270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768c109-76ba-4ccd-87bd-337fd8349b8e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:06.248037 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41dd621-6f9d-4822-a506-c882b44dcbb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:06.248760 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41dd621-6f9d-4822-a506-c882b44dcbb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:06.249037 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41dd621-6f9d-4822-a506-c882b44dcbb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:06.249309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41dd621-6f9d-4822-a506-c882b44dcbb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:06.259499 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41dd621-6f9d-4822-a506-c882b44dcbb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:06.260955 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41dd621-6f9d-4822-a506-c882b44dcbb6 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:06.270610 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1402/2830] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:27:06 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:27:06.444875 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fd360-c800-4f6c-a305-ef8995b5b021 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:06.459216 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fd360-c800-4f6c-a305-ef8995b5b021 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:06.464234 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f287a-02ce-4dd8-91d7-09abedd8e4ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:06.464834 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f287a-02ce-4dd8-91d7-09abedd8e4ee service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:06.465116 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f287a-02ce-4dd8-91d7-09abedd8e4ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:06.465382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f287a-02ce-4dd8-91d7-09abedd8e4ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:06.531464 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f287a-02ce-4dd8-91d7-09abedd8e4ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:06.532879 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f287a-02ce-4dd8-91d7-09abedd8e4ee service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:06.536546 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:06.544197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1429/2831] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:27:06 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:27:14.016876 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41dd621-6f9d-4822-a506-c882b44dcbb6 service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:14.082299 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41dd621-6f9d-4822-a506-c882b44dcbb6 service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:14.087285 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568dcbc-29b0-4cb9-87b7-488640515d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:14.087958 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568dcbc-29b0-4cb9-87b7-488640515d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:14.088231 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568dcbc-29b0-4cb9-87b7-488640515d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:14.088475 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568dcbc-29b0-4cb9-87b7-488640515d44 admin admin] PATH_INFO: `/v3/users/c6049d91d123462eac8bb251461c3354` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:14.089632 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568dcbc-29b0-4cb9-87b7-488640515d44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6049d91d123462eac8bb251461c3354)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:14.091596 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568dcbc-29b0-4cb9-87b7-488640515d44 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:14.134048 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:14.144854 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-8568dcbc-29b0-4cb9-87b7-488640515d44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6049d91d123462eac8bb251461c3354'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:14.145523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-8568dcbc-29b0-4cb9-87b7-488640515d44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6049d91d123462eac8bb251461c3354'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:14.146197 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-8568dcbc-29b0-4cb9-87b7-488640515d44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6049d91d123462eac8bb251461c3354'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:14.153577 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-8568dcbc-29b0-4cb9-87b7-488640515d44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6049d91d123462eac8bb251461c3354'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:14.165293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-8568dcbc-29b0-4cb9-87b7-488640515d44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6049d91d123462eac8bb251461c3354'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:14.179100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1403/2832] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:27:14 2022] DELETE /identity/v3/users/c6049d91d123462eac8bb251461c3354 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:14.183732 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f804b8e-ba2a-48a9-92d8-02f0003afca6 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:14.184231 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f804b8e-ba2a-48a9-92d8-02f0003afca6 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:14.184370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f804b8e-ba2a-48a9-92d8-02f0003afca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:14.204159 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-6f804b8e-ba2a-48a9-92d8-02f0003afca6 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:14.205371 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6f804b8e-ba2a-48a9-92d8-02f0003afca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:14.271071 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1430/2833] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:14.281189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568dcbc-29b0-4cb9-87b7-488640515d44 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:14.282599 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8568dcbc-29b0-4cb9-87b7-488640515d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:14.351668 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568dcbc-29b0-4cb9-87b7-488640515d44 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:14.356527 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cc0c6-6a68-412f-955f-00452775840b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:14.357265 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cc0c6-6a68-412f-955f-00452775840b service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:14.357544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cc0c6-6a68-412f-955f-00452775840b service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:14.357832 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cc0c6-6a68-412f-955f-00452775840b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:14.368914 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4cc0c6-6a68-412f-955f-00452775840b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:14.370366 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4cc0c6-6a68-412f-955f-00452775840b service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:14.380580 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1404/2834] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:27:14.656650 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f804b8e-ba2a-48a9-92d8-02f0003afca6 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:14.658218 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-6f804b8e-ba2a-48a9-92d8-02f0003afca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:14.673732 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f804b8e-ba2a-48a9-92d8-02f0003afca6 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:14.679257 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aac703-cf8d-4a46-90fb-7f0952b6d058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:14.679944 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aac703-cf8d-4a46-90fb-7f0952b6d058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:14.680221 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aac703-cf8d-4a46-90fb-7f0952b6d058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:14.681211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aac703-cf8d-4a46-90fb-7f0952b6d058 admin admin] PATH_INFO: `/v3/projects/08818811798f480e9b10300a0390ce80` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:14.682683 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aac703-cf8d-4a46-90fb-7f0952b6d058 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08818811798f480e9b10300a0390ce80)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:14.684658 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aac703-cf8d-4a46-90fb-7f0952b6d058 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:14.738038 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-52aac703-cf8d-4a46-90fb-7f0952b6d058 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08818811798f480e9b10300a0390ce80', 'request_id': 'req-52aac703-cf8d-4a46-90fb-7f0952b6d058'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:14.744083 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-52aac703-cf8d-4a46-90fb-7f0952b6d058 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08818811798f480e9b10300a0390ce80', 'request_id': 'req-52aac703-cf8d-4a46-90fb-7f0952b6d058'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:14.752129 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-52aac703-cf8d-4a46-90fb-7f0952b6d058 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08818811798f480e9b10300a0390ce80', 'request_id': 'req-52aac703-cf8d-4a46-90fb-7f0952b6d058'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:14.763393 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-52aac703-cf8d-4a46-90fb-7f0952b6d058 admin admin] The token cache is being invalidate because project 08818811798f480e9b10300a0390ce80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:27:14.763748 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-52aac703-cf8d-4a46-90fb-7f0952b6d058 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:14.764390 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-52aac703-cf8d-4a46-90fb-7f0952b6d058 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:14.765910 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1431/2835] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:27:14 2022] DELETE /identity/v3/projects/08818811798f480e9b10300a0390ce80 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:14.770167 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cc0c6-6a68-412f-955f-00452775840b service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:14.780152 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cc0c6-6a68-412f-955f-00452775840b service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:14.851100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f5889-62d3-4246-a381-aae9ae139892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:14.851782 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f5889-62d3-4246-a381-aae9ae139892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:14.852060 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f5889-62d3-4246-a381-aae9ae139892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:14.852332 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f5889-62d3-4246-a381-aae9ae139892 admin admin] PATH_INFO: `/v3/users/96744928634a452bb9c60999ee5c3792` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:14.853449 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f5889-62d3-4246-a381-aae9ae139892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96744928634a452bb9c60999ee5c3792)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:14.855445 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f5889-62d3-4246-a381-aae9ae139892 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:14.910501 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-668f5889-62d3-4246-a381-aae9ae139892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96744928634a452bb9c60999ee5c3792'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:14.911207 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-668f5889-62d3-4246-a381-aae9ae139892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96744928634a452bb9c60999ee5c3792'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:14.911856 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-668f5889-62d3-4246-a381-aae9ae139892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96744928634a452bb9c60999ee5c3792'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:14.922252 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-668f5889-62d3-4246-a381-aae9ae139892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96744928634a452bb9c60999ee5c3792'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:14.923926 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:14.943868 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-668f5889-62d3-4246-a381-aae9ae139892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96744928634a452bb9c60999ee5c3792'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:14.960005 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1405/2836] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:27:14 2022] DELETE /identity/v3/users/96744928634a452bb9c60999ee5c3792 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:15.286153 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aac703-cf8d-4a46-90fb-7f0952b6d058 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:15.288854 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-52aac703-cf8d-4a46-90fb-7f0952b6d058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:15.312473 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aac703-cf8d-4a46-90fb-7f0952b6d058 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:15.379843 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da1d07-e020-4eeb-afb1-22f08ffd6fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:15.380471 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da1d07-e020-4eeb-afb1-22f08ffd6fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:15.380748 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da1d07-e020-4eeb-afb1-22f08ffd6fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:15.381014 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da1d07-e020-4eeb-afb1-22f08ffd6fbb admin admin] PATH_INFO: `/v3/projects/4c3c25c09c244a289d1a524c20966997` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:15.382161 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da1d07-e020-4eeb-afb1-22f08ffd6fbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c3c25c09c244a289d1a524c20966997)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:15.384843 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da1d07-e020-4eeb-afb1-22f08ffd6fbb admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:15.414796 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:15.447255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-85da1d07-e020-4eeb-afb1-22f08ffd6fbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c3c25c09c244a289d1a524c20966997', 'request_id': 'req-85da1d07-e020-4eeb-afb1-22f08ffd6fbb'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:15.455015 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-85da1d07-e020-4eeb-afb1-22f08ffd6fbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c3c25c09c244a289d1a524c20966997', 'request_id': 'req-85da1d07-e020-4eeb-afb1-22f08ffd6fbb'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:15.465351 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-85da1d07-e020-4eeb-afb1-22f08ffd6fbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c3c25c09c244a289d1a524c20966997', 'request_id': 'req-85da1d07-e020-4eeb-afb1-22f08ffd6fbb'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:15.479828 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-85da1d07-e020-4eeb-afb1-22f08ffd6fbb admin admin] The token cache is being invalidate because project 4c3c25c09c244a289d1a524c20966997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:27:15.480258 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-85da1d07-e020-4eeb-afb1-22f08ffd6fbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:15.481336 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-85da1d07-e020-4eeb-afb1-22f08ffd6fbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:15.483105 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1432/2837] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:27:15 2022] DELETE /identity/v3/projects/4c3c25c09c244a289d1a524c20966997 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:15.489189 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a43cf3-0cdc-4c99-aa9e-5b6c226091d3 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:15.489721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a43cf3-0cdc-4c99-aa9e-5b6c226091d3 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:15.489721 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a43cf3-0cdc-4c99-aa9e-5b6c226091d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:15.513735 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-39a43cf3-0cdc-4c99-aa9e-5b6c226091d3 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:15.515384 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-39a43cf3-0cdc-4c99-aa9e-5b6c226091d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:15.597891 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1406/2838] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:15.606700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856ee4f-f93d-41dc-bef3-d9081aa3aa58 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:15.606978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856ee4f-f93d-41dc-bef3-d9081aa3aa58 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:15.607255 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856ee4f-f93d-41dc-bef3-d9081aa3aa58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:15.632074 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-5856ee4f-f93d-41dc-bef3-d9081aa3aa58 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:15.633289 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-5856ee4f-f93d-41dc-bef3-d9081aa3aa58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:15.645217 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1433/2839] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:15.651544 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a43cf3-0cdc-4c99-aa9e-5b6c226091d3 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:15.670759 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a43cf3-0cdc-4c99-aa9e-5b6c226091d3 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:15.678744 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b7a1c-99e1-4b42-ad32-2f3c6f4edb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:15.679373 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b7a1c-99e1-4b42-ad32-2f3c6f4edb3f admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:15.679681 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b7a1c-99e1-4b42-ad32-2f3c6f4edb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:15.679960 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b7a1c-99e1-4b42-ad32-2f3c6f4edb3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:15.680615 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b7a1c-99e1-4b42-ad32-2f3c6f4edb3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:27:15.680916 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b7a1c-99e1-4b42-ad32-2f3c6f4edb3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:15.682517 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b7a1c-99e1-4b42-ad32-2f3c6f4edb3f admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:15.699293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1407/2840] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:27:15.703999 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856ee4f-f93d-41dc-bef3-d9081aa3aa58 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:15.714970 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856ee4f-f93d-41dc-bef3-d9081aa3aa58 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:15.721165 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739a662-c663-4bea-b3e4-93fd7f253eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:15.721790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739a662-c663-4bea-b3e4-93fd7f253eca admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:15.722376 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739a662-c663-4bea-b3e4-93fd7f253eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:15.722667 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739a662-c663-4bea-b3e4-93fd7f253eca admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:15.723271 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739a662-c663-4bea-b3e4-93fd7f253eca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:15.725274 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739a662-c663-4bea-b3e4-93fd7f253eca admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:15.755630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1434/2841] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:27:15 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:27:15.761563 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b7a1c-99e1-4b42-ad32-2f3c6f4edb3f admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:15.777944 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b7a1c-99e1-4b42-ad32-2f3c6f4edb3f admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:15.863268 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e7c3b-1c50-4722-ad8c-53b282d8478d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:15.863845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e7c3b-1c50-4722-ad8c-53b282d8478d admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:15.864164 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e7c3b-1c50-4722-ad8c-53b282d8478d admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:15.864438 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e7c3b-1c50-4722-ad8c-53b282d8478d admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:15.865102 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e7c3b-1c50-4722-ad8c-53b282d8478d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:15.867088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e7c3b-1c50-4722-ad8c-53b282d8478d admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:15.890788 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1408/2842] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:27:15 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:27:15.897373 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739a662-c663-4bea-b3e4-93fd7f253eca admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:15.913211 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739a662-c663-4bea-b3e4-93fd7f253eca admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:15.919067 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ae67c-70ee-44ef-bc53-ac5988de7a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:15.919714 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ae67c-70ee-44ef-bc53-ac5988de7a6b admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:15.919987 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ae67c-70ee-44ef-bc53-ac5988de7a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:15.920247 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ae67c-70ee-44ef-bc53-ac5988de7a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:15.920896 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ae67c-70ee-44ef-bc53-ac5988de7a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:15.922524 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ae67c-70ee-44ef-bc53-ac5988de7a6b admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:15.933182 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1435/2843] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:27:15.937488 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e7c3b-1c50-4722-ad8c-53b282d8478d admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:15.949355 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e7c3b-1c50-4722-ad8c-53b282d8478d admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:15.954696 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146cb95-e256-46bf-9dbe-fcb807f020dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:15.955325 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146cb95-e256-46bf-9dbe-fcb807f020dd admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:15.955687 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146cb95-e256-46bf-9dbe-fcb807f020dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:15.955966 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146cb95-e256-46bf-9dbe-fcb807f020dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:15.956621 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146cb95-e256-46bf-9dbe-fcb807f020dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:15.958278 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146cb95-e256-46bf-9dbe-fcb807f020dd admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:15.970825 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1409/2844] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:27:15.977664 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ae67c-70ee-44ef-bc53-ac5988de7a6b admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:15.989623 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ae67c-70ee-44ef-bc53-ac5988de7a6b admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:15.995114 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8027f-f81a-4358-ad77-8a9e04aee445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:15.995837 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8027f-f81a-4358-ad77-8a9e04aee445 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:15.996090 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8027f-f81a-4358-ad77-8a9e04aee445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:15.996362 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8027f-f81a-4358-ad77-8a9e04aee445 admin admin] PATH_INFO: `/v3/projects/e89b898ae0a5425a9b78b6c41f1e8874/users/4f53587918094789a0a37f3bf8cb4dd5/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:16.012434 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8027f-f81a-4358-ad77-8a9e04aee445 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e89b898ae0a5425a9b78b6c41f1e8874, user_id=4f53587918094789a0a37f3bf8cb4dd5, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:16.015192 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8027f-f81a-4358-ad77-8a9e04aee445 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:16.034357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1436/2845] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:27:15 2022] PUT /identity/v3/projects/e89b898ae0a5425a9b78b6c41f1e8874/users/4f53587918094789a0a37f3bf8cb4dd5/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:16.045379 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b050407-0037-4076-a994-ada4a20eeb0b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:16.045741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b050407-0037-4076-a994-ada4a20eeb0b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:16.046008 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b050407-0037-4076-a994-ada4a20eeb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:16.075545 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-6b050407-0037-4076-a994-ada4a20eeb0b None None] MFA Rules not processed for user `4f53587918094789a0a37f3bf8cb4dd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:16.076806 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-6b050407-0037-4076-a994-ada4a20eeb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:16.117453 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:16.155744 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1410/2846] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:16.160651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8027f-f81a-4358-ad77-8a9e04aee445 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:16.234056 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8027f-f81a-4358-ad77-8a9e04aee445 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:16.241477 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e353dc-015b-4e22-9261-31f3b9139da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:16.242109 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e353dc-015b-4e22-9261-31f3b9139da4 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:16.242359 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e353dc-015b-4e22-9261-31f3b9139da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:16.242626 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e353dc-015b-4e22-9261-31f3b9139da4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:16.243243 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e353dc-015b-4e22-9261-31f3b9139da4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:16.245345 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e353dc-015b-4e22-9261-31f3b9139da4 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:16.281399 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1437/2847] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:27:16 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:27:16.286174 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b050407-0037-4076-a994-ada4a20eeb0b None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:16.297579 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b050407-0037-4076-a994-ada4a20eeb0b None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:16.355976 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ce7c1-0fb8-4334-8c9d-93700ba71926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:16.356617 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ce7c1-0fb8-4334-8c9d-93700ba71926 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:16.357017 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ce7c1-0fb8-4334-8c9d-93700ba71926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:16.357293 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ce7c1-0fb8-4334-8c9d-93700ba71926 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:16.357947 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ce7c1-0fb8-4334-8c9d-93700ba71926 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:16.359939 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ce7c1-0fb8-4334-8c9d-93700ba71926 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:16.382367 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1411/2848] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:27:16 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:27:16.387315 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e353dc-015b-4e22-9261-31f3b9139da4 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:16.400042 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e353dc-015b-4e22-9261-31f3b9139da4 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:16.405959 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdfd793-df43-442b-bc7a-482a5d7e630d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:16.406585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfd793-df43-442b-bc7a-482a5d7e630d admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:16.406862 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfd793-df43-442b-bc7a-482a5d7e630d admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:16.407135 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfd793-df43-442b-bc7a-482a5d7e630d admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:16.407881 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdfd793-df43-442b-bc7a-482a5d7e630d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:16.409518 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdfd793-df43-442b-bc7a-482a5d7e630d admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:16.420203 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1438/2849] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:27:16.424678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ce7c1-0fb8-4334-8c9d-93700ba71926 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:16.434940 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ce7c1-0fb8-4334-8c9d-93700ba71926 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:16.439724 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766855e-9d97-402f-b67c-a7d09283bd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:16.440431 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766855e-9d97-402f-b67c-a7d09283bd1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:16.440708 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766855e-9d97-402f-b67c-a7d09283bd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:16.440991 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766855e-9d97-402f-b67c-a7d09283bd1e admin admin] PATH_INFO: `/v3/projects/abd8ed5e98374a229ccaeff5237ea277/users/f753fc07c5734472af1653c7e6b722d3/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:16.455994 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766855e-9d97-402f-b67c-a7d09283bd1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=abd8ed5e98374a229ccaeff5237ea277, user_id=f753fc07c5734472af1653c7e6b722d3, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:16.458686 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766855e-9d97-402f-b67c-a7d09283bd1e admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:16.474202 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1412/2850] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:27:16 2022] PUT /identity/v3/projects/abd8ed5e98374a229ccaeff5237ea277/users/f753fc07c5734472af1653c7e6b722d3/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:16.478370 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdfd793-df43-442b-bc7a-482a5d7e630d admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:16.492483 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdfd793-df43-442b-bc7a-482a5d7e630d admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:16.515517 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:16.558263 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56c458-55ef-462e-9372-47e28ef128de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:16.558900 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56c458-55ef-462e-9372-47e28ef128de admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:16.559180 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56c458-55ef-462e-9372-47e28ef128de admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:16.559480 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56c458-55ef-462e-9372-47e28ef128de admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:16.560156 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56c458-55ef-462e-9372-47e28ef128de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:16.562088 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56c458-55ef-462e-9372-47e28ef128de admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:16.573002 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1439/2851] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:27:16.577624 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766855e-9d97-402f-b67c-a7d09283bd1e admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:16.588699 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766855e-9d97-402f-b67c-a7d09283bd1e admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:16.594405 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51cbe3-251f-4c8d-97b9-26f67e595373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:16.595078 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51cbe3-251f-4c8d-97b9-26f67e595373 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:16.595351 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51cbe3-251f-4c8d-97b9-26f67e595373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:16.595660 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51cbe3-251f-4c8d-97b9-26f67e595373 admin admin] PATH_INFO: `/v3/projects/abd8ed5e98374a229ccaeff5237ea277/users/f753fc07c5734472af1653c7e6b722d3/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:16.598255 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51cbe3-251f-4c8d-97b9-26f67e595373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abd8ed5e98374a229ccaeff5237ea277, user_id=f753fc07c5734472af1653c7e6b722d3, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:16.600975 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51cbe3-251f-4c8d-97b9-26f67e595373 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:16.611908 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1413/2852] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:27:16 2022] PUT /identity/v3/projects/abd8ed5e98374a229ccaeff5237ea277/users/f753fc07c5734472af1653c7e6b722d3/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:16.623263 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3609c0-2587-4d43-b2fb-4c190be5ac63 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:16.623581 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3609c0-2587-4d43-b2fb-4c190be5ac63 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:16.623858 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3609c0-2587-4d43-b2fb-4c190be5ac63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:16.653014 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-8b3609c0-2587-4d43-b2fb-4c190be5ac63 None None] MFA Rules not processed for user `f753fc07c5734472af1653c7e6b722d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:16.654208 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8b3609c0-2587-4d43-b2fb-4c190be5ac63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:16.734503 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1440/2853] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:16.747088 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51cbe3-251f-4c8d-97b9-26f67e595373 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:16.749341 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8a51cbe3-251f-4c8d-97b9-26f67e595373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:16.825560 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51cbe3-251f-4c8d-97b9-26f67e595373 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:16.829701 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a1919-3d20-4423-aa1c-aa8ae1335523 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:16.830314 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a1919-3d20-4423-aa1c-aa8ae1335523 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:16.830839 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a1919-3d20-4423-aa1c-aa8ae1335523 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:16.830981 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a1919-3d20-4423-aa1c-aa8ae1335523 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:16.840727 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a1919-3d20-4423-aa1c-aa8ae1335523 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:16.842162 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a1919-3d20-4423-aa1c-aa8ae1335523 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:16.852083 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1414/2854] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:27:16 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:27:16.932316 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3609c0-2587-4d43-b2fb-4c190be5ac63 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:16.943256 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3609c0-2587-4d43-b2fb-4c190be5ac63 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:16.947651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f9f8e-1db9-45c8-b320-5f353d3dbcf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:16.948261 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f9f8e-1db9-45c8-b320-5f353d3dbcf6 service cinder] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:16.948534 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f9f8e-1db9-45c8-b320-5f353d3dbcf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:16.948802 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f9f8e-1db9-45c8-b320-5f353d3dbcf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:17.010381 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f9f8e-1db9-45c8-b320-5f353d3dbcf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:17.011868 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f9f8e-1db9-45c8-b320-5f353d3dbcf6 service cinder] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:17.021731 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1441/2855] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:27:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:27:17.313576 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a1919-3d20-4423-aa1c-aa8ae1335523 service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:17.341778 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:17.381242 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a1919-3d20-4423-aa1c-aa8ae1335523 service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:17.386143 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f9dd7-811a-4888-ad95-bb99cfea581d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:17.386769 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f9dd7-811a-4888-ad95-bb99cfea581d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:17.387042 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f9dd7-811a-4888-ad95-bb99cfea581d admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:17.387306 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f9dd7-811a-4888-ad95-bb99cfea581d admin admin] PATH_INFO: `/v3/users/4f53587918094789a0a37f3bf8cb4dd5` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:17.388490 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f9dd7-811a-4888-ad95-bb99cfea581d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f53587918094789a0a37f3bf8cb4dd5)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:17.390465 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f9dd7-811a-4888-ad95-bb99cfea581d admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:17.451380 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-632f9dd7-811a-4888-ad95-bb99cfea581d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f53587918094789a0a37f3bf8cb4dd5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:17.452155 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-632f9dd7-811a-4888-ad95-bb99cfea581d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f53587918094789a0a37f3bf8cb4dd5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:17.452810 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-632f9dd7-811a-4888-ad95-bb99cfea581d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f53587918094789a0a37f3bf8cb4dd5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:17.460791 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-632f9dd7-811a-4888-ad95-bb99cfea581d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f53587918094789a0a37f3bf8cb4dd5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:17.473915 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-632f9dd7-811a-4888-ad95-bb99cfea581d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f53587918094789a0a37f3bf8cb4dd5'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:17.486412 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1415/2856] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:27:17 2022] DELETE /identity/v3/users/4f53587918094789a0a37f3bf8cb4dd5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:17.491444 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35674273-e8db-4450-b278-1f17ad382da5 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:17.491951 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35674273-e8db-4450-b278-1f17ad382da5 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:17.492080 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35674273-e8db-4450-b278-1f17ad382da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:17.512651 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-35674273-e8db-4450-b278-1f17ad382da5 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:17.513864 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-35674273-e8db-4450-b278-1f17ad382da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:17.555960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=100 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:17.582711 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1442/2857] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:17.592549 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f9dd7-811a-4888-ad95-bb99cfea581d admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:17.594113 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-632f9dd7-811a-4888-ad95-bb99cfea581d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:17.657566 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f9dd7-811a-4888-ad95-bb99cfea581d admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:17.662062 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29f4a2-4ac7-400b-8b9b-dec3210cf0fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:17.662676 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29f4a2-4ac7-400b-8b9b-dec3210cf0fa service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:17.662950 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29f4a2-4ac7-400b-8b9b-dec3210cf0fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:17.663223 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29f4a2-4ac7-400b-8b9b-dec3210cf0fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:17.672899 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29f4a2-4ac7-400b-8b9b-dec3210cf0fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:17.674399 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29f4a2-4ac7-400b-8b9b-dec3210cf0fa service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:17.684543 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1416/2858] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:27:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:27:17.977803 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35674273-e8db-4450-b278-1f17ad382da5 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:17.979330 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-35674273-e8db-4450-b278-1f17ad382da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:17.992133 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35674273-e8db-4450-b278-1f17ad382da5 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:17.997700 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db4d1b-0ac7-4385-a9e9-5dfc2ed57194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:17.998357 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db4d1b-0ac7-4385-a9e9-5dfc2ed57194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:17.998638 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db4d1b-0ac7-4385-a9e9-5dfc2ed57194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:17.998906 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db4d1b-0ac7-4385-a9e9-5dfc2ed57194 admin admin] PATH_INFO: `/v3/projects/e89b898ae0a5425a9b78b6c41f1e8874` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:18.000184 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db4d1b-0ac7-4385-a9e9-5dfc2ed57194 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e89b898ae0a5425a9b78b6c41f1e8874)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:18.002147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db4d1b-0ac7-4385-a9e9-5dfc2ed57194 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:18.054008 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-61db4d1b-0ac7-4385-a9e9-5dfc2ed57194 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e89b898ae0a5425a9b78b6c41f1e8874', 'request_id': 'req-61db4d1b-0ac7-4385-a9e9-5dfc2ed57194'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:18.060013 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-61db4d1b-0ac7-4385-a9e9-5dfc2ed57194 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e89b898ae0a5425a9b78b6c41f1e8874', 'request_id': 'req-61db4d1b-0ac7-4385-a9e9-5dfc2ed57194'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:18.069356 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-61db4d1b-0ac7-4385-a9e9-5dfc2ed57194 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e89b898ae0a5425a9b78b6c41f1e8874', 'request_id': 'req-61db4d1b-0ac7-4385-a9e9-5dfc2ed57194'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:18.087262 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-61db4d1b-0ac7-4385-a9e9-5dfc2ed57194 admin admin] The token cache is being invalidate because project e89b898ae0a5425a9b78b6c41f1e8874 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:27:18.087790 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-61db4d1b-0ac7-4385-a9e9-5dfc2ed57194 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:18.088886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-61db4d1b-0ac7-4385-a9e9-5dfc2ed57194 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:18.091454 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1443/2859] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:27:17 2022] DELETE /identity/v3/projects/e89b898ae0a5425a9b78b6c41f1e8874 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:18.098003 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29f4a2-4ac7-400b-8b9b-dec3210cf0fa service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:18.108309 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29f4a2-4ac7-400b-8b9b-dec3210cf0fa service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:18.153777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:18.170552 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81cac40-2729-4f15-805e-d886403a2bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:18.171190 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cac40-2729-4f15-805e-d886403a2bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:18.171500 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cac40-2729-4f15-805e-d886403a2bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:18.171774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cac40-2729-4f15-805e-d886403a2bfe admin admin] PATH_INFO: `/v3/users/f753fc07c5734472af1653c7e6b722d3` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:18.172935 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81cac40-2729-4f15-805e-d886403a2bfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=f753fc07c5734472af1653c7e6b722d3)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:18.175516 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81cac40-2729-4f15-805e-d886403a2bfe admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:18.223237 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-a81cac40-2729-4f15-805e-d886403a2bfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f753fc07c5734472af1653c7e6b722d3'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:18.223926 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-a81cac40-2729-4f15-805e-d886403a2bfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f753fc07c5734472af1653c7e6b722d3'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:18.224577 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-a81cac40-2729-4f15-805e-d886403a2bfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f753fc07c5734472af1653c7e6b722d3'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:18.231890 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-a81cac40-2729-4f15-805e-d886403a2bfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f753fc07c5734472af1653c7e6b722d3'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:18.246246 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-a81cac40-2729-4f15-805e-d886403a2bfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f753fc07c5734472af1653c7e6b722d3'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:18.263846 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1417/2860] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:27:18 2022] DELETE /identity/v3/users/f753fc07c5734472af1653c7e6b722d3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:18.553210 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db4d1b-0ac7-4385-a9e9-5dfc2ed57194 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:18.554785 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-61db4d1b-0ac7-4385-a9e9-5dfc2ed57194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:18.568755 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db4d1b-0ac7-4385-a9e9-5dfc2ed57194 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:18.627559 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfbb23-0f22-4470-9721-59aef05dd7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:18.628206 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfbb23-0f22-4470-9721-59aef05dd7b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:18.628487 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfbb23-0f22-4470-9721-59aef05dd7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:18.628761 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfbb23-0f22-4470-9721-59aef05dd7b9 admin admin] PATH_INFO: `/v3/projects/abd8ed5e98374a229ccaeff5237ea277` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:18.629890 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfbb23-0f22-4470-9721-59aef05dd7b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abd8ed5e98374a229ccaeff5237ea277)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:18.631882 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfbb23-0f22-4470-9721-59aef05dd7b9 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:18.679435 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-05bfbb23-0f22-4470-9721-59aef05dd7b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abd8ed5e98374a229ccaeff5237ea277', 'request_id': 'req-05bfbb23-0f22-4470-9721-59aef05dd7b9'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:18.684912 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-05bfbb23-0f22-4470-9721-59aef05dd7b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abd8ed5e98374a229ccaeff5237ea277', 'request_id': 'req-05bfbb23-0f22-4470-9721-59aef05dd7b9'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:18.693569 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-05bfbb23-0f22-4470-9721-59aef05dd7b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abd8ed5e98374a229ccaeff5237ea277', 'request_id': 'req-05bfbb23-0f22-4470-9721-59aef05dd7b9'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:18.695631 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:18.706429 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-05bfbb23-0f22-4470-9721-59aef05dd7b9 admin admin] The token cache is being invalidate because project abd8ed5e98374a229ccaeff5237ea277 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81715) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:27:18.706955 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-05bfbb23-0f22-4470-9721-59aef05dd7b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:18.707621 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-05bfbb23-0f22-4470-9721-59aef05dd7b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:18.709124 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1444/2861] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:27:18 2022] DELETE /identity/v3/projects/abd8ed5e98374a229ccaeff5237ea277 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:18.714485 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57abe3-f1a9-40b5-857d-0b05daaea578 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:18.714972 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57abe3-f1a9-40b5-857d-0b05daaea578 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:18.715100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57abe3-f1a9-40b5-857d-0b05daaea578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:18.739152 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-bc57abe3-f1a9-40b5-857d-0b05daaea578 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:18.740412 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-bc57abe3-f1a9-40b5-857d-0b05daaea578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:18.808683 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1418/2862] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:18.817467 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca2bfc-16d4-4dab-ab36-fdb23d88ad87 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:18.817592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca2bfc-16d4-4dab-ab36-fdb23d88ad87 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:18.817893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca2bfc-16d4-4dab-ab36-fdb23d88ad87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:18.839336 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-66ca2bfc-16d4-4dab-ab36-fdb23d88ad87 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:18.840577 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-66ca2bfc-16d4-4dab-ab36-fdb23d88ad87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:18.850664 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1445/2863] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 07:27:18.855131 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57abe3-f1a9-40b5-857d-0b05daaea578 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:18.867203 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57abe3-f1a9-40b5-857d-0b05daaea578 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:18.874145 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25abd7c1-055a-418f-849d-15e0591fb80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:18.874975 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abd7c1-055a-418f-849d-15e0591fb80c admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:18.875240 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abd7c1-055a-418f-849d-15e0591fb80c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:18.875689 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abd7c1-055a-418f-849d-15e0591fb80c admin admin] PATH_INFO: `/v3/domains` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:18.876301 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25abd7c1-055a-418f-849d-15e0591fb80c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:27:18.876601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25abd7c1-055a-418f-849d-15e0591fb80c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:18.878457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25abd7c1-055a-418f-849d-15e0591fb80c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:18.894212 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1419/2864] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:27:18.899163 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca2bfc-16d4-4dab-ab36-fdb23d88ad87 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:18.909562 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca2bfc-16d4-4dab-ab36-fdb23d88ad87 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:18.915568 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9e58f-f6d9-427f-aa48-8ee489c340f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:18.916160 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9e58f-f6d9-427f-aa48-8ee489c340f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:18.916433 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9e58f-f6d9-427f-aa48-8ee489c340f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:18.916698 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9e58f-f6d9-427f-aa48-8ee489c340f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:18.917386 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf9e58f-f6d9-427f-aa48-8ee489c340f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:18.919294 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf9e58f-f6d9-427f-aa48-8ee489c340f6 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:18.957010 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1446/2865] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:27:18 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:27:18.961844 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25abd7c1-055a-418f-849d-15e0591fb80c admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:18.975385 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25abd7c1-055a-418f-849d-15e0591fb80c admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:19.027006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:19.039810 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaacca4-02e1-4431-973e-39962b8f56e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:19.040422 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaacca4-02e1-4431-973e-39962b8f56e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:19.040703 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaacca4-02e1-4431-973e-39962b8f56e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:19.040975 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaacca4-02e1-4431-973e-39962b8f56e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:19.041642 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaacca4-02e1-4431-973e-39962b8f56e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:19.043628 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaacca4-02e1-4431-973e-39962b8f56e2 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:19.067009 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1420/2866] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:27:18 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:27:19.071297 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9e58f-f6d9-427f-aa48-8ee489c340f6 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:19.082095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9e58f-f6d9-427f-aa48-8ee489c340f6 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:19.087608 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03a0e7-435e-4bfe-a295-3dd4f9f10da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:19.088206 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03a0e7-435e-4bfe-a295-3dd4f9f10da6 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:19.088481 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03a0e7-435e-4bfe-a295-3dd4f9f10da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:19.088748 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03a0e7-435e-4bfe-a295-3dd4f9f10da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:19.089382 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e03a0e7-435e-4bfe-a295-3dd4f9f10da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:19.090978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e03a0e7-435e-4bfe-a295-3dd4f9f10da6 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:19.101649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1447/2867] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:27:19.107125 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaacca4-02e1-4431-973e-39962b8f56e2 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:19.118360 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaacca4-02e1-4431-973e-39962b8f56e2 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:19.123589 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a629eb-eaa7-4ee3-a2b6-a3f38417e612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:19.124187 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a629eb-eaa7-4ee3-a2b6-a3f38417e612 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:19.124462 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a629eb-eaa7-4ee3-a2b6-a3f38417e612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:19.124716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a629eb-eaa7-4ee3-a2b6-a3f38417e612 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:19.125354 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a629eb-eaa7-4ee3-a2b6-a3f38417e612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:19.126952 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a629eb-eaa7-4ee3-a2b6-a3f38417e612 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:19.138202 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1421/2868] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:27:19.142506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03a0e7-435e-4bfe-a295-3dd4f9f10da6 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:19.152674 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03a0e7-435e-4bfe-a295-3dd4f9f10da6 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:19.157773 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702f42b-a3c6-4f7b-ade5-d075b95ace36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:19.158425 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702f42b-a3c6-4f7b-ade5-d075b95ace36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:19.158801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702f42b-a3c6-4f7b-ade5-d075b95ace36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:19.159121 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702f42b-a3c6-4f7b-ade5-d075b95ace36 admin admin] PATH_INFO: `/v3/projects/9537cbc63f654d01badb23a6804ab135/users/3e33f9ea7b3546f5926431a523c40ac3/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:19.173906 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702f42b-a3c6-4f7b-ade5-d075b95ace36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9537cbc63f654d01badb23a6804ab135, user_id=3e33f9ea7b3546f5926431a523c40ac3, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:19.177108 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702f42b-a3c6-4f7b-ade5-d075b95ace36 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:19.192083 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1448/2869] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:27:19 2022] PUT /identity/v3/projects/9537cbc63f654d01badb23a6804ab135/users/3e33f9ea7b3546f5926431a523c40ac3/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:19.206457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e63a2-6fee-478c-b7d6-6c3e3185c510 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:19.206693 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e63a2-6fee-478c-b7d6-6c3e3185c510 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:19.207008 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e63a2-6fee-478c-b7d6-6c3e3185c510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:19.241374 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-ab5e63a2-6fee-478c-b7d6-6c3e3185c510 None None] MFA Rules not processed for user `3e33f9ea7b3546f5926431a523c40ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:19.242590 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-ab5e63a2-6fee-478c-b7d6-6c3e3185c510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:19.313113 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1422/2870] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:19.318225 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702f42b-a3c6-4f7b-ade5-d075b95ace36 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:19.386178 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702f42b-a3c6-4f7b-ade5-d075b95ace36 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:19.391341 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fb5ee-f35c-457f-b115-6e16a4f02c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:19.391912 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fb5ee-f35c-457f-b115-6e16a4f02c01 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:19.392191 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fb5ee-f35c-457f-b115-6e16a4f02c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:19.392459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fb5ee-f35c-457f-b115-6e16a4f02c01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:19.393124 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51fb5ee-f35c-457f-b115-6e16a4f02c01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:19.395039 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51fb5ee-f35c-457f-b115-6e16a4f02c01 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:19.425464 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1449/2871] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:27:19 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:27:19.429668 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e63a2-6fee-478c-b7d6-6c3e3185c510 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:19.440857 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e63a2-6fee-478c-b7d6-6c3e3185c510 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:19.500818 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf4850-7a56-42b0-b97b-283ddf2fbbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:19.501430 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf4850-7a56-42b0-b97b-283ddf2fbbf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:19.501706 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf4850-7a56-42b0-b97b-283ddf2fbbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:19.501977 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf4850-7a56-42b0-b97b-283ddf2fbbf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:19.502649 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf4850-7a56-42b0-b97b-283ddf2fbbf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:19.504700 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf4850-7a56-42b0-b97b-283ddf2fbbf0 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:19.528988 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1423/2872] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:27:19 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:27:19.534678 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fb5ee-f35c-457f-b115-6e16a4f02c01 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:19.550544 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fb5ee-f35c-457f-b115-6e16a4f02c01 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:19.559556 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8bf28-f6b8-4c35-b1c9-db0c44621f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:19.560197 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8bf28-f6b8-4c35-b1c9-db0c44621f9a admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:19.560428 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8bf28-f6b8-4c35-b1c9-db0c44621f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:19.560697 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8bf28-f6b8-4c35-b1c9-db0c44621f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:19.561338 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8bf28-f6b8-4c35-b1c9-db0c44621f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:19.562941 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8bf28-f6b8-4c35-b1c9-db0c44621f9a admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:19.574677 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1450/2873] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:27:19.579038 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf4850-7a56-42b0-b97b-283ddf2fbbf0 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:19.590383 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf4850-7a56-42b0-b97b-283ddf2fbbf0 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:19.598059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa4f8a-33f4-4b2a-841c-e7a3554a3c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:19.598715 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa4f8a-33f4-4b2a-841c-e7a3554a3c22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:19.599006 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa4f8a-33f4-4b2a-841c-e7a3554a3c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:19.599270 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa4f8a-33f4-4b2a-841c-e7a3554a3c22 admin admin] PATH_INFO: `/v3/projects/774041ec05654a1192e870fb49015af6/users/0b7536903a6d45cdb24e0d450b8a25c3/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:19.615593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa4f8a-33f4-4b2a-841c-e7a3554a3c22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=774041ec05654a1192e870fb49015af6, user_id=0b7536903a6d45cdb24e0d450b8a25c3, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:19.618273 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa4f8a-33f4-4b2a-841c-e7a3554a3c22 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:19.634251 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1424/2874] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:27:19 2022] PUT /identity/v3/projects/774041ec05654a1192e870fb49015af6/users/0b7536903a6d45cdb24e0d450b8a25c3/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:19.638334 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8bf28-f6b8-4c35-b1c9-db0c44621f9a admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:19.649561 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8bf28-f6b8-4c35-b1c9-db0c44621f9a admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:19.712100 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d46a6-3d19-45dd-9bb0-eeaf3a7b37ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:19.712718 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d46a6-3d19-45dd-9bb0-eeaf3a7b37ba admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:19.712995 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d46a6-3d19-45dd-9bb0-eeaf3a7b37ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:19.713269 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d46a6-3d19-45dd-9bb0-eeaf3a7b37ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:19.713904 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d46a6-3d19-45dd-9bb0-eeaf3a7b37ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:19.715531 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d46a6-3d19-45dd-9bb0-eeaf3a7b37ba admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:19.726585 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1451/2875] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:27:19.731020 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa4f8a-33f4-4b2a-841c-e7a3554a3c22 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:19.741996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa4f8a-33f4-4b2a-841c-e7a3554a3c22 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:19.747234 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948c41e-47fe-46d5-b85c-0e39e856053b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:19.747914 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948c41e-47fe-46d5-b85c-0e39e856053b admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:19.748192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948c41e-47fe-46d5-b85c-0e39e856053b admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:19.748464 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948c41e-47fe-46d5-b85c-0e39e856053b admin admin] PATH_INFO: `/v3/projects/774041ec05654a1192e870fb49015af6/users/0b7536903a6d45cdb24e0d450b8a25c3/roles/2fda19ac4eb940169d7e1ef16c60a45f` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:19.750381 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948c41e-47fe-46d5-b85c-0e39e856053b admin admin] RBAC: Authorizing `identity:create_grant(project_id=774041ec05654a1192e870fb49015af6, user_id=0b7536903a6d45cdb24e0d450b8a25c3, role_id=2fda19ac4eb940169d7e1ef16c60a45f)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:19.753715 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948c41e-47fe-46d5-b85c-0e39e856053b admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:19.763946 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1425/2876] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:27:19 2022] PUT /identity/v3/projects/774041ec05654a1192e870fb49015af6/users/0b7536903a6d45cdb24e0d450b8a25c3/roles/2fda19ac4eb940169d7e1ef16c60a45f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:19.774640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84b7cd-b3e8-4cc3-9c67-bbf7e6225f2e None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:19.774926 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84b7cd-b3e8-4cc3-9c67-bbf7e6225f2e None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:19.775231 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84b7cd-b3e8-4cc3-9c67-bbf7e6225f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:19.782686 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:19.806117 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-8e84b7cd-b3e8-4cc3-9c67-bbf7e6225f2e None None] MFA Rules not processed for user `0b7536903a6d45cdb24e0d450b8a25c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:19.807320 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-8e84b7cd-b3e8-4cc3-9c67-bbf7e6225f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:19.884734 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1452/2877] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:19.898397 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948c41e-47fe-46d5-b85c-0e39e856053b admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:19.901207 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-8948c41e-47fe-46d5-b85c-0e39e856053b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:19.922932 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:19.971559 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948c41e-47fe-46d5-b85c-0e39e856053b admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:19.975848 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c30aa-231e-4a41-b13c-50f4cdd3a784 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:19.976457 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c30aa-231e-4a41-b13c-50f4cdd3a784 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:19.976728 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c30aa-231e-4a41-b13c-50f4cdd3a784 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:19.976996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c30aa-231e-4a41-b13c-50f4cdd3a784 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:20.033192 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c30aa-231e-4a41-b13c-50f4cdd3a784 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:20.034680 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c30aa-231e-4a41-b13c-50f4cdd3a784 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:20.043629 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1426/2878] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:27:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:27:22.788809 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84b7cd-b3e8-4cc3-9c67-bbf7e6225f2e None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:22.876244 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84b7cd-b3e8-4cc3-9c67-bbf7e6225f2e None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:22.881968 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ec3aa-bd05-4107-bef4-a07917d7c0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:22.882592 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ec3aa-bd05-4107-bef4-a07917d7c0dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:22.882874 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ec3aa-bd05-4107-bef4-a07917d7c0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:22.883135 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ec3aa-bd05-4107-bef4-a07917d7c0dd admin admin] PATH_INFO: `/v3/users/3e33f9ea7b3546f5926431a523c40ac3` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:22.884307 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ec3aa-bd05-4107-bef4-a07917d7c0dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e33f9ea7b3546f5926431a523c40ac3)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:22.886241 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ec3aa-bd05-4107-bef4-a07917d7c0dd admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:22.942059 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5d6ec3aa-bd05-4107-bef4-a07917d7c0dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e33f9ea7b3546f5926431a523c40ac3'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:22.942783 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5d6ec3aa-bd05-4107-bef4-a07917d7c0dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e33f9ea7b3546f5926431a523c40ac3'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:22.943438 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5d6ec3aa-bd05-4107-bef4-a07917d7c0dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e33f9ea7b3546f5926431a523c40ac3'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:22.950338 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5d6ec3aa-bd05-4107-bef4-a07917d7c0dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e33f9ea7b3546f5926431a523c40ac3'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:22.961942 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-5d6ec3aa-bd05-4107-bef4-a07917d7c0dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e33f9ea7b3546f5926431a523c40ac3'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:22.975485 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1453/2879] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:27:22 2022] DELETE /identity/v3/users/3e33f9ea7b3546f5926431a523c40ac3 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:22.980139 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89380948-b2b5-477c-9e86-8ca2e98cd78d None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:22.980647 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89380948-b2b5-477c-9e86-8ca2e98cd78d None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:22.980777 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89380948-b2b5-477c-9e86-8ca2e98cd78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:23.000291 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-89380948-b2b5-477c-9e86-8ca2e98cd78d None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:23.001458 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-89380948-b2b5-477c-9e86-8ca2e98cd78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:23.070313 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1427/2880] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:23.081413 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ec3aa-bd05-4107-bef4-a07917d7c0dd admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:23.083553 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-5d6ec3aa-bd05-4107-bef4-a07917d7c0dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:23.101524 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:23.171536 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ec3aa-bd05-4107-bef4-a07917d7c0dd admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:23.175762 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f5f4e-de0a-464c-908f-ac4a99310f6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:23.176398 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f5f4e-de0a-464c-908f-ac4a99310f6d service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:23.176666 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f5f4e-de0a-464c-908f-ac4a99310f6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:23.176939 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f5f4e-de0a-464c-908f-ac4a99310f6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:23.186565 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f5f4e-de0a-464c-908f-ac4a99310f6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:23.188091 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f5f4e-de0a-464c-908f-ac4a99310f6d service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:23.198097 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1454/2881] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:27:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 07:27:23.472930 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89380948-b2b5-477c-9e86-8ca2e98cd78d None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:23.474414 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-89380948-b2b5-477c-9e86-8ca2e98cd78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:23.487287 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89380948-b2b5-477c-9e86-8ca2e98cd78d None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:23.493741 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa61a7-5a11-4067-ab3b-f2b629f07d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:23.494392 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa61a7-5a11-4067-ab3b-f2b629f07d74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:23.494667 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa61a7-5a11-4067-ab3b-f2b629f07d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:23.494934 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa61a7-5a11-4067-ab3b-f2b629f07d74 admin admin] PATH_INFO: `/v3/projects/9537cbc63f654d01badb23a6804ab135` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:23.496125 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa61a7-5a11-4067-ab3b-f2b629f07d74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9537cbc63f654d01badb23a6804ab135)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:23.498184 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa61a7-5a11-4067-ab3b-f2b629f07d74 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:23.552821 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-adaa61a7-5a11-4067-ab3b-f2b629f07d74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9537cbc63f654d01badb23a6804ab135', 'request_id': 'req-adaa61a7-5a11-4067-ab3b-f2b629f07d74'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:23.559277 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-adaa61a7-5a11-4067-ab3b-f2b629f07d74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9537cbc63f654d01badb23a6804ab135', 'request_id': 'req-adaa61a7-5a11-4067-ab3b-f2b629f07d74'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:23.563205 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:23.569493 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-adaa61a7-5a11-4067-ab3b-f2b629f07d74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9537cbc63f654d01badb23a6804ab135', 'request_id': 'req-adaa61a7-5a11-4067-ab3b-f2b629f07d74'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:23.582513 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-adaa61a7-5a11-4067-ab3b-f2b629f07d74 admin admin] The token cache is being invalidate because project 9537cbc63f654d01badb23a6804ab135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:27:23.582798 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-adaa61a7-5a11-4067-ab3b-f2b629f07d74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:23.583463 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-adaa61a7-5a11-4067-ab3b-f2b629f07d74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:23.585058 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1428/2882] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:27:23 2022] DELETE /identity/v3/projects/9537cbc63f654d01badb23a6804ab135 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:23.589311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f5f4e-de0a-464c-908f-ac4a99310f6d service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:23.600022 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f5f4e-de0a-464c-908f-ac4a99310f6d service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:23.663757 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa2424-204c-4d97-a6a3-3c3f851e9d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:23.664436 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa2424-204c-4d97-a6a3-3c3f851e9d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:23.664710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa2424-204c-4d97-a6a3-3c3f851e9d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:23.664979 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa2424-204c-4d97-a6a3-3c3f851e9d76 admin admin] PATH_INFO: `/v3/users/0b7536903a6d45cdb24e0d450b8a25c3` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:23.666334 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa2424-204c-4d97-a6a3-3c3f851e9d76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b7536903a6d45cdb24e0d450b8a25c3)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:23.668254 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa2424-204c-4d97-a6a3-3c3f851e9d76 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:23.717667 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-14aa2424-204c-4d97-a6a3-3c3f851e9d76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b7536903a6d45cdb24e0d450b8a25c3'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:23.718345 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-14aa2424-204c-4d97-a6a3-3c3f851e9d76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b7536903a6d45cdb24e0d450b8a25c3'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:23.718960 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-14aa2424-204c-4d97-a6a3-3c3f851e9d76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b7536903a6d45cdb24e0d450b8a25c3'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:23.726779 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-14aa2424-204c-4d97-a6a3-3c3f851e9d76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b7536903a6d45cdb24e0d450b8a25c3'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:23.739649 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-14aa2424-204c-4d97-a6a3-3c3f851e9d76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b7536903a6d45cdb24e0d450b8a25c3'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:23.754294 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1455/2883] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:27:23 2022] DELETE /identity/v3/users/0b7536903a6d45cdb24e0d450b8a25c3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:24.028080 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa61a7-5a11-4067-ab3b-f2b629f07d74 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:24.029757 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-adaa61a7-5a11-4067-ab3b-f2b629f07d74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:24.044706 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa61a7-5a11-4067-ab3b-f2b629f07d74 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:24.106049 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43c0ee-ee18-49a7-995e-1170edc6c361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:24.106684 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43c0ee-ee18-49a7-995e-1170edc6c361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:24.106984 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43c0ee-ee18-49a7-995e-1170edc6c361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:24.107281 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43c0ee-ee18-49a7-995e-1170edc6c361 admin admin] PATH_INFO: `/v3/projects/774041ec05654a1192e870fb49015af6` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:24.108512 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43c0ee-ee18-49a7-995e-1170edc6c361 admin admin] RBAC: Authorizing `identity:delete_project(project_id=774041ec05654a1192e870fb49015af6)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:24.110470 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43c0ee-ee18-49a7-995e-1170edc6c361 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:24.163201 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-5b43c0ee-ee18-49a7-995e-1170edc6c361 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '774041ec05654a1192e870fb49015af6', 'request_id': 'req-5b43c0ee-ee18-49a7-995e-1170edc6c361'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:24.169603 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-5b43c0ee-ee18-49a7-995e-1170edc6c361 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '774041ec05654a1192e870fb49015af6', 'request_id': 'req-5b43c0ee-ee18-49a7-995e-1170edc6c361'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:24.179034 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-5b43c0ee-ee18-49a7-995e-1170edc6c361 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '774041ec05654a1192e870fb49015af6', 'request_id': 'req-5b43c0ee-ee18-49a7-995e-1170edc6c361'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:24.194845 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-5b43c0ee-ee18-49a7-995e-1170edc6c361 admin admin] The token cache is being invalidate because project 774041ec05654a1192e870fb49015af6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:27:24.195100 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-5b43c0ee-ee18-49a7-995e-1170edc6c361 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:24.195875 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-5b43c0ee-ee18-49a7-995e-1170edc6c361 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:27:24.197533 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1429/2884] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:27:24 2022] DELETE /identity/v3/projects/774041ec05654a1192e870fb49015af6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:24.202665 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18a73a-9248-4ce5-9b4e-446e0ffca449 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:24.203251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18a73a-9248-4ce5-9b4e-446e0ffca449 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:24.203251 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18a73a-9248-4ce5-9b4e-446e0ffca449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:24.223853 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-4f18a73a-9248-4ce5-9b4e-446e0ffca449 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:24.225076 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-4f18a73a-9248-4ce5-9b4e-446e0ffca449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:24.255426 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:24.302095 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1456/2885] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:24.310304 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be22cc-0b61-4c58-b004-88b66c7969b9 None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:24.310572 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be22cc-0b61-4c58-b004-88b66c7969b9 None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:24.310862 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be22cc-0b61-4c58-b004-88b66c7969b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:24.332837 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-f4be22cc-0b61-4c58-b004-88b66c7969b9 None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:24.334103 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-f4be22cc-0b61-4c58-b004-88b66c7969b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:24.343837 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1430/2886] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:24.349806 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18a73a-9248-4ce5-9b4e-446e0ffca449 None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:24.364839 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18a73a-9248-4ce5-9b4e-446e0ffca449 None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:24.371147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a1151-e6a7-4f9a-b976-daa648e3b665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:24.371801 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a1151-e6a7-4f9a-b976-daa648e3b665 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:24.372072 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a1151-e6a7-4f9a-b976-daa648e3b665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:24.372345 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a1151-e6a7-4f9a-b976-daa648e3b665 admin admin] PATH_INFO: `/v3/domains` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:24.372991 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a1151-e6a7-4f9a-b976-daa648e3b665 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81715) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:27:24.373295 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a1151-e6a7-4f9a-b976-daa648e3b665 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:24.374902 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a1151-e6a7-4f9a-b976-daa648e3b665 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:24.387965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1457/2887] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Aug 23 07:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:27:24.392378 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be22cc-0b61-4c58-b004-88b66c7969b9 None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:24.402783 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be22cc-0b61-4c58-b004-88b66c7969b9 None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:24.408685 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ef569-825f-4ad9-ab3d-fa084b8c254c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:24.409772 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ef569-825f-4ad9-ab3d-fa084b8c254c admin admin] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:24.410054 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ef569-825f-4ad9-ab3d-fa084b8c254c admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:24.410324 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ef569-825f-4ad9-ab3d-fa084b8c254c admin admin] PATH_INFO: `/v3/projects` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:24.410948 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ef569-825f-4ad9-ab3d-fa084b8c254c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:24.412902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ef569-825f-4ad9-ab3d-fa084b8c254c admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:24.443653 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1431/2888] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Aug 23 07:27:24 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:27:24.447897 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a1151-e6a7-4f9a-b976-daa648e3b665 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:24.458904 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a1151-e6a7-4f9a-b976-daa648e3b665 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:24.530096 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb61317-74b4-4856-98f6-31eb25148da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:24.530704 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb61317-74b4-4856-98f6-31eb25148da5 admin admin] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:24.530983 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb61317-74b4-4856-98f6-31eb25148da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:24.531262 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb61317-74b4-4856-98f6-31eb25148da5 admin admin] PATH_INFO: `/v3/users` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:24.531918 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb61317-74b4-4856-98f6-31eb25148da5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:24.533848 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb61317-74b4-4856-98f6-31eb25148da5 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:24.560904 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1458/2889] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Aug 23 07:27:24 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:27:24.566026 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ef569-825f-4ad9-ab3d-fa084b8c254c admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:24.582296 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ef569-825f-4ad9-ab3d-fa084b8c254c admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:24.587987 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb0e27-b80f-4dbc-ae51-849aeb717022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:24.588588 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb0e27-b80f-4dbc-ae51-849aeb717022 admin admin] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:24.588866 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb0e27-b80f-4dbc-ae51-849aeb717022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:24.589140 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb0e27-b80f-4dbc-ae51-849aeb717022 admin admin] PATH_INFO: `/v3/roles` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:24.589774 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb0e27-b80f-4dbc-ae51-849aeb717022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:24.591433 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb0e27-b80f-4dbc-ae51-849aeb717022 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:24.602055 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1432/2890] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:27:24.606144 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb61317-74b4-4856-98f6-31eb25148da5 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:24.616511 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb61317-74b4-4856-98f6-31eb25148da5 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:24.621738 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5b346-4d59-4d38-a6c0-b4a4487463c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:24.622337 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5b346-4d59-4d38-a6c0-b4a4487463c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:24.622621 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5b346-4d59-4d38-a6c0-b4a4487463c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:24.622889 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5b346-4d59-4d38-a6c0-b4a4487463c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:24.623547 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5b346-4d59-4d38-a6c0-b4a4487463c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:24.625147 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5b346-4d59-4d38-a6c0-b4a4487463c5 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:24.635774 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1459/2891] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Aug 23 07:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:27:24.642097 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb0e27-b80f-4dbc-ae51-849aeb717022 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:24.652762 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb0e27-b80f-4dbc-ae51-849aeb717022 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:24.658066 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103cec4-65db-4a06-a921-f3e23b00f181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:24.658714 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103cec4-65db-4a06-a921-f3e23b00f181 admin admin] REQUEST_METHOD: `PUT` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:24.659059 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103cec4-65db-4a06-a921-f3e23b00f181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:24.659333 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103cec4-65db-4a06-a921-f3e23b00f181 admin admin] PATH_INFO: `/v3/projects/5b02989040ab4080855af2e66c942060/users/ac00970326df432bb130e44ed04db3b5/roles/7670c7c450b9404ba57e12743cb6c458` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:24.675820 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103cec4-65db-4a06-a921-f3e23b00f181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b02989040ab4080855af2e66c942060, user_id=ac00970326df432bb130e44ed04db3b5, role_id=7670c7c450b9404ba57e12743cb6c458)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:24.678511 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103cec4-65db-4a06-a921-f3e23b00f181 admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:24.693027 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1433/2892] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Aug 23 07:27:24 2022] PUT /identity/v3/projects/5b02989040ab4080855af2e66c942060/users/ac00970326df432bb130e44ed04db3b5/roles/7670c7c450b9404ba57e12743cb6c458 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:27:24.698199 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f4556-efcf-4401-a8a5-fcc9acd2b75c None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:24.698407 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f4556-efcf-4401-a8a5-fcc9acd2b75c None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:24.698680 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f4556-efcf-4401-a8a5-fcc9acd2b75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:24.717641 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-b53f4556-efcf-4401-a8a5-fcc9acd2b75c None None] MFA Rules not processed for user `2ff4f07bfac14f428ece0dfbc5ec3579`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:24.718838 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b53f4556-efcf-4401-a8a5-fcc9acd2b75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:24.777965 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1460/2893] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:24.789475 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103cec4-65db-4a06-a921-f3e23b00f181 admin admin] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:24.791386 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-5103cec4-65db-4a06-a921-f3e23b00f181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:24.813075 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:24.859218 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103cec4-65db-4a06-a921-f3e23b00f181 admin admin] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:24.863690 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde56c2-60b5-4495-8010-aa2025ffe2e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:24.864311 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde56c2-60b5-4495-8010-aa2025ffe2e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:24.864577 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde56c2-60b5-4495-8010-aa2025ffe2e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:24.864849 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde56c2-60b5-4495-8010-aa2025ffe2e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:24.873894 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde56c2-60b5-4495-8010-aa2025ffe2e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:24.875377 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde56c2-60b5-4495-8010-aa2025ffe2e3 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:24.886557 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1434/2894] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:27:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:27:25.507771 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396f492-a459-45a1-a604-e2a97d5764c9 None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:25.508292 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396f492-a459-45a1-a604-e2a97d5764c9 None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:25.508446 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396f492-a459-45a1-a604-e2a97d5764c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:25.538696 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-5396f492-a459-45a1-a604-e2a97d5764c9 None None] MFA Rules not processed for user `ac00970326df432bb130e44ed04db3b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:25.539949 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-5396f492-a459-45a1-a604-e2a97d5764c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:25.564750 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:25.612913 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1461/2895] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:25.624090 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde56c2-60b5-4495-8010-aa2025ffe2e3 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:25.625678 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2dde56c2-60b5-4495-8010-aa2025ffe2e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:25.690752 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde56c2-60b5-4495-8010-aa2025ffe2e3 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:25.694618 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bb471-4402-4144-8865-9a045bce8bc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a26f09e892a74d9eab97e73cce9d062a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:25.695332 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bb471-4402-4144-8865-9a045bce8bc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:25.695637 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bb471-4402-4144-8865-9a045bce8bc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:25.695904 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bb471-4402-4144-8865-9a045bce8bc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:25.704286 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bb471-4402-4144-8865-9a045bce8bc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:25.705738 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bb471-4402-4144-8865-9a045bce8bc1 service cinder] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:25.714767 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1435/2896] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:27:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:27:27.084820 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f90f81-ce7a-4462-a6eb-4783fb399aee None None] REQUEST_METHOD: `POST` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:27.085358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f90f81-ce7a-4462-a6eb-4783fb399aee None None] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:27.085657 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f90f81-ce7a-4462-a6eb-4783fb399aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:27.114216 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.auth.core [None req-71f90f81-ce7a-4462-a6eb-4783fb399aee None None] MFA Rules not processed for user `ac00970326df432bb130e44ed04db3b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81715) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:27:27.116029 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-71f90f81-ce7a-4462-a6eb-4783fb399aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:27.127297 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1462/2897] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:27:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:27:27.376596 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bb471-4402-4144-8865-9a045bce8bc1 service cinder] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:27.388523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bb471-4402-4144-8865-9a045bce8bc1 service cinder] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:27.394099 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70e418-e6b5-4320-9719-208be02c7f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:27.394716 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70e418-e6b5-4320-9719-208be02c7f34 service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:27.395010 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70e418-e6b5-4320-9719-208be02c7f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:27.395261 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70e418-e6b5-4320-9719-208be02c7f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:27.397372 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-df70e418-e6b5-4320-9719-208be02c7f34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:27.408032 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df70e418-e6b5-4320-9719-208be02c7f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:27.409859 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df70e418-e6b5-4320-9719-208be02c7f34 service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:27.418200 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1436/2898] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:27:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:27:33.573891 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f90f81-ce7a-4462-a6eb-4783fb399aee None None] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:33.585358 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f90f81-ce7a-4462-a6eb-4783fb399aee None None] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:33.589493 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7eb3d8-66a5-4d59-82f1-ed16dc99c93a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:33.590090 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7eb3d8-66a5-4d59-82f1-ed16dc99c93a service neutron] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:33.590387 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7eb3d8-66a5-4d59-82f1-ed16dc99c93a service neutron] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:33.590630 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7eb3d8-66a5-4d59-82f1-ed16dc99c93a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:33.592521 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ac7eb3d8-66a5-4d59-82f1-ed16dc99c93a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:33.667893 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7eb3d8-66a5-4d59-82f1-ed16dc99c93a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:33.669388 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7eb3d8-66a5-4d59-82f1-ed16dc99c93a service neutron] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:33.682151 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1463/2899] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:27:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:27:34.880538 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70e418-e6b5-4320-9719-208be02c7f34 service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:27:34.889893 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70e418-e6b5-4320-9719-208be02c7f34 service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:27:34.896049 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bc77c-7118-430c-96f4-c4bda971e67f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:27:34.896839 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bc77c-7118-430c-96f4-c4bda971e67f service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:27:34.898241 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bc77c-7118-430c-96f4-c4bda971e67f service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:27:34.899455 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bc77c-7118-430c-96f4-c4bda971e67f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:27:34.902291 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-807bc77c-7118-430c-96f4-c4bda971e67f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:27:34.919758 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807bc77c-7118-430c-96f4-c4bda971e67f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:27:34.921128 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807bc77c-7118-430c-96f4-c4bda971e67f service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:27:34.932902 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1437/2900] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:27:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:27:43.675710 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=53 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:27:44.927867 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=56 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:28:15.168287 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7eb3d8-66a5-4d59-82f1-ed16dc99c93a service neutron] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:28:15.170232 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-ac7eb3d8-66a5-4d59-82f1-ed16dc99c93a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:28:15.186712 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7eb3d8-66a5-4d59-82f1-ed16dc99c93a service neutron] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:28:15.193743 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3eefc-97d4-4dbb-9ff3-fa4db43f6623 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8d2cd93042c4583b63afe629d47fce0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:28:15.194364 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3eefc-97d4-4dbb-9ff3-fa4db43f6623 service nova] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:28:15.194640 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3eefc-97d4-4dbb-9ff3-fa4db43f6623 service nova] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:28:15.194901 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3eefc-97d4-4dbb-9ff3-fa4db43f6623 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:28:15.197006 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-daf3eefc-97d4-4dbb-9ff3-fa4db43f6623 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:28:15.209861 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3eefc-97d4-4dbb-9ff3-fa4db43f6623 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:28:15.211457 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3eefc-97d4-4dbb-9ff3-fa4db43f6623 service nova] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:28:15.223834 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1464/2901] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:28:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:28:24.552607 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d381cba-9db5-4498-a2a3-75d9b8a0154b None None] REQUEST_METHOD: `POST` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:28:24.553160 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d381cba-9db5-4498-a2a3-75d9b8a0154b None None] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:28:24.553160 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d381cba-9db5-4498-a2a3-75d9b8a0154b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:28:24.584167 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.auth.core [None req-2d381cba-9db5-4498-a2a3-75d9b8a0154b None None] MFA Rules not processed for user `ac00970326df432bb130e44ed04db3b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:28:24.586126 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2d381cba-9db5-4498-a2a3-75d9b8a0154b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:28:24.598033 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1438/2902] 10.222.0.17 () {58 vars in 958 bytes} [Tue Aug 23 07:28:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 07:28:25.169060 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3eefc-97d4-4dbb-9ff3-fa4db43f6623 service nova] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:28:25.170506 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-daf3eefc-97d4-4dbb-9ff3-fa4db43f6623 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:28:25.240978 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3eefc-97d4-4dbb-9ff3-fa4db43f6623 service nova] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:28:25.246436 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7179aac-63f7-4531-b60f-27de4de2ac85 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9b0f54c988e43bda474f978a69fa4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:28:25.247053 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7179aac-63f7-4531-b60f-27de4de2ac85 service placement] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:28:25.247328 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7179aac-63f7-4531-b60f-27de4de2ac85 service placement] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:28:25.247627 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7179aac-63f7-4531-b60f-27de4de2ac85 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:28:25.249417 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b7179aac-63f7-4531-b60f-27de4de2ac85 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:28:25.262138 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7179aac-63f7-4531-b60f-27de4de2ac85 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:28:25.263786 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7179aac-63f7-4531-b60f-27de4de2ac85 service placement] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:28:25.273546 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1465/2903] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:28:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:28:26.890046 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d381cba-9db5-4498-a2a3-75d9b8a0154b None None] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:28:26.892229 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-2d381cba-9db5-4498-a2a3-75d9b8a0154b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:28:26.905609 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d381cba-9db5-4498-a2a3-75d9b8a0154b None None] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:28:26.909353 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cad36e-6182-49a0-a5a5-5f1571f2741c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '938364c02d8d4dd2ae832c5129be3cbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:28:26.910127 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cad36e-6182-49a0-a5a5-5f1571f2741c service neutron] REQUEST_METHOD: `GET` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:28:26.910523 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cad36e-6182-49a0-a5a5-5f1571f2741c service neutron] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:28:26.910795 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cad36e-6182-49a0-a5a5-5f1571f2741c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:28:26.912601 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.fernet_utils [None req-d0cad36e-6182-49a0-a5a5-5f1571f2741c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:28:26.922310 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cad36e-6182-49a0-a5a5-5f1571f2741c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:28:26.923766 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cad36e-6182-49a0-a5a5-5f1571f2741c service neutron] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:28:26.932158 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1439/2904] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:28:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:28:35.269562 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=38 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:28:36.929134 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=9 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:29:17.270002 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7179aac-63f7-4531-b60f-27de4de2ac85 service placement] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:29:17.281550 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7179aac-63f7-4531-b60f-27de4de2ac85 service placement] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:29:17.288023 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d9aa7-56f3-4538-aa50-9122ef4d8817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:29:17.288709 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d9aa7-56f3-4538-aa50-9122ef4d8817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:29:17.289011 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d9aa7-56f3-4538-aa50-9122ef4d8817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:29:17.289302 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d9aa7-56f3-4538-aa50-9122ef4d8817 admin admin] PATH_INFO: `/v3/users/ac00970326df432bb130e44ed04db3b5` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:29:17.290544 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d9aa7-56f3-4538-aa50-9122ef4d8817 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac00970326df432bb130e44ed04db3b5)` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:29:17.292594 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d9aa7-56f3-4538-aa50-9122ef4d8817 admin admin] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:29:17.342805 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-b10d9aa7-56f3-4538-aa50-9122ef4d8817 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac00970326df432bb130e44ed04db3b5'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:29:17.343593 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-b10d9aa7-56f3-4538-aa50-9122ef4d8817 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac00970326df432bb130e44ed04db3b5'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:29:17.344312 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-b10d9aa7-56f3-4538-aa50-9122ef4d8817 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac00970326df432bb130e44ed04db3b5'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:29:17.351822 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-b10d9aa7-56f3-4538-aa50-9122ef4d8817 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac00970326df432bb130e44ed04db3b5'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:29:17.364574 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.notifications [None req-b10d9aa7-56f3-4538-aa50-9122ef4d8817 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac00970326df432bb130e44ed04db3b5'} {{(pid=81715) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:29:17.380141 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1466/2905] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Aug 23 07:29:17 2022] DELETE /identity/v3/users/ac00970326df432bb130e44ed04db3b5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:29:17.824593 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cad36e-6182-49a0-a5a5-5f1571f2741c service neutron] Authenticating user token {{(pid=81714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:29:17.837609 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cad36e-6182-49a0-a5a5-5f1571f2741c service neutron] Validating token access rules against request {{(pid=81714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:29:17.896691 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a900a07-6d61-4d03-bece-684e5d9a9afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ff4f07bfac14f428ece0dfbc5ec3579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7014afbcdaa44f68b1b75e344808f41', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:29:17.897338 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a900a07-6d61-4d03-bece-684e5d9a9afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:29:17.897614 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a900a07-6d61-4d03-bece-684e5d9a9afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:29:17.897888 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a900a07-6d61-4d03-bece-684e5d9a9afc admin admin] PATH_INFO: `/v3/projects/5b02989040ab4080855af2e66c942060` {{(pid=81714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:29:17.899022 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a900a07-6d61-4d03-bece-684e5d9a9afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b02989040ab4080855af2e66c942060)` {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:29:17.900996 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a900a07-6d61-4d03-bece-684e5d9a9afc admin admin] RBAC: Authorization granted {{(pid=81714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:29:17.948665 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4a900a07-6d61-4d03-bece-684e5d9a9afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b02989040ab4080855af2e66c942060', 'request_id': 'req-4a900a07-6d61-4d03-bece-684e5d9a9afc'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:29:17.954392 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4a900a07-6d61-4d03-bece-684e5d9a9afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b02989040ab4080855af2e66c942060', 'request_id': 'req-4a900a07-6d61-4d03-bece-684e5d9a9afc'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:29:17.964907 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4a900a07-6d61-4d03-bece-684e5d9a9afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b02989040ab4080855af2e66c942060', 'request_id': 'req-4a900a07-6d61-4d03-bece-684e5d9a9afc'} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:29:17.979394 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4a900a07-6d61-4d03-bece-684e5d9a9afc admin admin] The token cache is being invalidate because project 5b02989040ab4080855af2e66c942060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 07:29:17.979879 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4a900a07-6d61-4d03-bece-684e5d9a9afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:29:17.980791 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG keystone.notifications [None req-4a900a07-6d61-4d03-bece-684e5d9a9afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 07:29:17.983066 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: [pid: 81714|app: 0|req: 1440/2906] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Aug 23 07:29:17 2022] DELETE /identity/v3/projects/5b02989040ab4080855af2e66c942060 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:29:27.375993 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:29:27.974731 dv-cinder-854092-1-iscsi devstack@keystone.service[81714]: DEBUG dbcounter [-] [81714] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:31:42.090763 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d9aa7-56f3-4538-aa50-9122ef4d8817 admin admin] Authenticating user token {{(pid=81715) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:31:42.092469 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-b10d9aa7-56f3-4538-aa50-9122ef4d8817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:31:42.239789 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d9aa7-56f3-4538-aa50-9122ef4d8817 admin admin] Validating token access rules against request {{(pid=81715) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:31:42.305788 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913154e2-e46f-4078-af86-50b6d28c261e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9b0f54c988e43bda474f978a69fa4fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '351a6777525c4207826d7df0d3768e99', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81715) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:31:42.306311 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913154e2-e46f-4078-af86-50b6d28c261e service placement] REQUEST_METHOD: `GET` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:31:42.306588 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913154e2-e46f-4078-af86-50b6d28c261e service placement] SCRIPT_NAME: `/identity` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:31:42.306886 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913154e2-e46f-4078-af86-50b6d28c261e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81715) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:31:42.308746 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.fernet_utils [None req-913154e2-e46f-4078-af86-50b6d28c261e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81715) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:31:42.320459 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913154e2-e46f-4078-af86-50b6d28c261e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:31:42.322050 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913154e2-e46f-4078-af86-50b6d28c261e service placement] RBAC: Authorization granted {{(pid=81715) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:31:42.332225 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: [pid: 81715|app: 0|req: 1467/2907] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Aug 23 07:31:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 07:31:52.328122 dv-cinder-854092-1-iscsi devstack@keystone.service[81715]: DEBUG dbcounter [-] [81715] Writing DB stats keystone:SELECT=89 {{(pid=81715) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}